[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:41:08 dist-test-slave-7kzw krb5kdc[5021](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:41:08 dist-test-slave-7kzw krb5kdc[5021](info): set up 2 sockets
Aug 10 18:41:08 dist-test-slave-7kzw krb5kdc[5021](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260810 18:41:09.534224 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.449s user 0.002s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:41:09 dist-test-slave-7kzw krb5kdc[5021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:41:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:41:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.228.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
I20260810 18:41:09.792425 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:38651
--webserver_interface=127.4.228.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:37833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:38651
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:41:10.499468 5040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:41:10.500034 5040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:41:10.500319 5040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:41:10.510947 5040 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:41:10.511207 5040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:41:10.511364 5040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:41:10.511631 5040 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:41:10.511751 5040 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:41:10.526770 5040 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:37833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:38651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:38651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:41:10.529054 5040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:41:10.531311 5040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:41:10.557440 5053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:10.557687 5049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:10.557664 5050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:10.559861 5040 server_base.cc:1034] running on GCE node
I20260810 18:41:10.561359 5040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:41:10.564138 5040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:41:10.565501 5040 hybrid_clock.cc:648] HybridClock initialized: now 1786387270565364 us; error 121 us; skew 500 ppm
I20260810 18:41:10.566371 5040 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:41:10 dist-test-slave-7kzw krb5kdc[5021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387270, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:41:10.572615 5040 init.cc:377] Logged in from keytab as kudu/127.4.228.254@KRBTEST.COM (short username kudu)
I20260810 18:41:10.576468 5040 webserver.cc:460] Webserver started at http://127.4.228.254:39743/ using document root <none> and password file <none>
I20260810 18:41:10.577920 5040 fs_manager.cc:359] Metadata directory not provided
I20260810 18:41:10.578173 5040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:41:10.578881 5040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:41:10.584573 5040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "27924b44336e4f8e8d2e22ab0d918e32"
format_stamp: "Formatted at 2026-08-10 18:41:10 on dist-test-slave-7kzw"
server_key: "28de5aea50526480a6ca2f49cc0d936d"
server_key_iv: "4365bcb226820b44ab32c9db4493eb71"
server_key_version: "encryptionkey@0"
I20260810 18:41:10.585628 5040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "27924b44336e4f8e8d2e22ab0d918e32"
format_stamp: "Formatted at 2026-08-10 18:41:10 on dist-test-slave-7kzw"
server_key: "28de5aea50526480a6ca2f49cc0d936d"
server_key_iv: "4365bcb226820b44ab32c9db4493eb71"
server_key_version: "encryptionkey@0"
I20260810 18:41:10.596189 5040 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.011s sys 0.000s
I20260810 18:41:10.604135 5066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:10.606988 5040 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.003s sys 0.003s
I20260810 18:41:10.607486 5040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "27924b44336e4f8e8d2e22ab0d918e32"
format_stamp: "Formatted at 2026-08-10 18:41:10 on dist-test-slave-7kzw"
server_key: "28de5aea50526480a6ca2f49cc0d936d"
server_key_iv: "4365bcb226820b44ab32c9db4493eb71"
server_key_version: "encryptionkey@0"
I20260810 18:41:10.607875 5040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:41:10.631896 5040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:41:10.640784 5040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:41:10.641366 5040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:41:10.698767 5040 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:38651
I20260810 18:41:10.698846 5140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:38651 every 8 connection(s)
I20260810 18:41:10.701788 5040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:41:10.709002 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 5040
I20260810 18:41:10.709645 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:41:10.712692 5141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:10.713928 5011 external_mini_cluster.cc:1428] Setting key 02f470c07a784eaa8ce00563e627b947
I20260810 18:41:10.738313 5141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27924b44336e4f8e8d2e22ab0d918e32: Bootstrap starting.
I20260810 18:41:10.748354 5141 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27924b44336e4f8e8d2e22ab0d918e32: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:10.751925 5141 log.cc:824] T 00000000000000000000000000000000 P 27924b44336e4f8e8d2e22ab0d918e32: Log is configured to *not* fsync() on all Append() calls
I20260810 18:41:10.759291 5141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27924b44336e4f8e8d2e22ab0d918e32: No bootstrap required, opened a new log
Aug 10 18:41:10 dist-test-slave-7kzw krb5kdc[5021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:10.775378 5141 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27924b44336e4f8e8d2e22ab0d918e32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27924b44336e4f8e8d2e22ab0d918e32" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 38651 } }
I20260810 18:41:10.776515 5141 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27924b44336e4f8e8d2e22ab0d918e32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:10.776846 5141 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27924b44336e4f8e8d2e22ab0d918e32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27924b44336e4f8e8d2e22ab0d918e32, State: Initialized, Role: FOLLOWER
I20260810 18:41:10.779017 5141 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27924b44336e4f8e8d2e22ab0d918e32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27924b44336e4f8e8d2e22ab0d918e32" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 38651 } }
I20260810 18:41:10.779668 5141 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27924b44336e4f8e8d2e22ab0d918e32 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:41:10.780068 5141 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27924b44336e4f8e8d2e22ab0d918e32 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:41:10.780414 5141 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27924b44336e4f8e8d2e22ab0d918e32 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:10.785135 5141 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27924b44336e4f8e8d2e22ab0d918e32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27924b44336e4f8e8d2e22ab0d918e32" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 38651 } }
I20260810 18:41:10.786522 5141 leader_election.cc:302] T 00000000000000000000000000000000 P 27924b44336e4f8e8d2e22ab0d918e32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27924b44336e4f8e8d2e22ab0d918e32; no voters:
I20260810 18:41:10.788219 5141 leader_election.cc:288] T 00000000000000000000000000000000 P 27924b44336e4f8e8d2e22ab0d918e32 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:41:10.788460 5152 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27924b44336e4f8e8d2e22ab0d918e32 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:41:10.788563 5148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:10.723990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41800 (local address 127.4.228.254:38651)
0810 18:41:10.725967 (+ 1977us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:10.725992 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:10.730323 (+ 4331us) server_negotiation.cc:407] Connection header received
0810 18:41:10.747816 (+ 17493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:10.747859 (+ 43us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:10.748955 (+ 1096us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:10.749932 (+ 977us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:10.755448 (+ 5516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:10.758770 (+ 3322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:10.761575 (+ 2805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:10.763251 (+ 1676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:10.763597 (+ 346us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:10.771589 (+ 7992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:10.771674 (+ 85us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:10.771752 (+ 78us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:10.771840 (+ 88us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:10.779437 (+ 7597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:10.780968 (+ 1531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:10.780980 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:10.780988 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:10.781334 (+ 346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:10.781995 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:10.782002 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:10.782004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:10.783483 (+ 1479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:10.784002 (+ 519us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:10.785131 (+ 1129us) server_negotiation.cc:299] Negotiation successful
0810 18:41:10.785684 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1534,"thread_start_us":879,"threads_started":1}
I20260810 18:41:10.790884 5152 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27924b44336e4f8e8d2e22ab0d918e32 [term 1 LEADER]: Becoming Leader. State: Replica: 27924b44336e4f8e8d2e22ab0d918e32, State: Running, Role: LEADER
I20260810 18:41:10.792258 5152 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27924b44336e4f8e8d2e22ab0d918e32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27924b44336e4f8e8d2e22ab0d918e32" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 38651 } }
I20260810 18:41:10.793875 5141 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27924b44336e4f8e8d2e22ab0d918e32 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:41:10.805576 5155 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27924b44336e4f8e8d2e22ab0d918e32 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27924b44336e4f8e8d2e22ab0d918e32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27924b44336e4f8e8d2e22ab0d918e32" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 38651 } } }
I20260810 18:41:10.805469 5157 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27924b44336e4f8e8d2e22ab0d918e32 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27924b44336e4f8e8d2e22ab0d918e32. Latest consensus state: current_term: 1 leader_uuid: "27924b44336e4f8e8d2e22ab0d918e32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27924b44336e4f8e8d2e22ab0d918e32" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 38651 } } }
I20260810 18:41:10.806792 5155 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27924b44336e4f8e8d2e22ab0d918e32 [sys.catalog]: This master's current role is: LEADER
I20260810 18:41:10.806793 5157 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27924b44336e4f8e8d2e22ab0d918e32 [sys.catalog]: This master's current role is: LEADER
W20260810 18:41:10.829471 5181 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 27924b44336e4f8e8d2e22ab0d918e32: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:41:10.829694 5181 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:41:10.830025 5174 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:41:10.834734 5174 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:41:10.857149 5174 catalog_manager.cc:1347] Generated new cluster ID: 2f65da94c9d14b0380024c77d0835832
I20260810 18:41:10.857396 5174 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:41:10.888903 5174 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:41:10.891880 5174 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:41:10.903751 5174 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27924b44336e4f8e8d2e22ab0d918e32: Generated new TSK 0
I20260810 18:41:10.906288 5174 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.228.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab.
Entry for principal kudu/127.4.228.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab.
I20260810 18:41:10.985402 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.193:0
--local_ip_for_outbound_sockets=127.4.228.193
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:38651
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:37833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:41:11.667384 5188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:41:11.667974 5188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:41:11.668267 5188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:41:11.679693 5188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:41:11.679889 5188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:41:11.680068 5188 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.193
I20260810 18:41:11.693023 5188 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:37833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:38651
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.193
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:41:11.694686 5188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:41:11.696796 5188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:41:11.717762 5195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:11.718768 5200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:11.718983 5197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:11.721141 5188 server_base.cc:1034] running on GCE node
I20260810 18:41:11.721995 5188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:41:11.723562 5188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:41:11.724874 5188 hybrid_clock.cc:648] HybridClock initialized: now 1786387271724755 us; error 59 us; skew 500 ppm
I20260810 18:41:11.725402 5188 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:41:11 dist-test-slave-7kzw krb5kdc[5021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387271, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:41:11.730770 5188 init.cc:377] Logged in from keytab as kudu/127.4.228.193@KRBTEST.COM (short username kudu)
I20260810 18:41:11.733310 5188 webserver.cc:460] Webserver started at http://127.4.228.193:37741/ using document root <none> and password file <none>
I20260810 18:41:11.734360 5188 fs_manager.cc:359] Metadata directory not provided
I20260810 18:41:11.734634 5188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:41:11.735083 5188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:41:11.739264 5188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-0/data/instance:
uuid: "bcf5ab79a8c74ff399f7ad7da31e709a"
format_stamp: "Formatted at 2026-08-10 18:41:11 on dist-test-slave-7kzw"
server_key: "61b9319b3c87900b33a6e1f5ce612c07"
server_key_iv: "30773b4bc087a7c054a71c0433ca56d9"
server_key_version: "encryptionkey@0"
I20260810 18:41:11.740331 5188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance:
uuid: "bcf5ab79a8c74ff399f7ad7da31e709a"
format_stamp: "Formatted at 2026-08-10 18:41:11 on dist-test-slave-7kzw"
server_key: "61b9319b3c87900b33a6e1f5ce612c07"
server_key_iv: "30773b4bc087a7c054a71c0433ca56d9"
server_key_version: "encryptionkey@0"
I20260810 18:41:11.747905 5188 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.002s
I20260810 18:41:11.753105 5209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:11.754871 5188 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260810 18:41:11.755156 5188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-0/wal
uuid: "bcf5ab79a8c74ff399f7ad7da31e709a"
format_stamp: "Formatted at 2026-08-10 18:41:11 on dist-test-slave-7kzw"
server_key: "61b9319b3c87900b33a6e1f5ce612c07"
server_key_iv: "30773b4bc087a7c054a71c0433ca56d9"
server_key_version: "encryptionkey@0"
I20260810 18:41:11.755486 5188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:41:11.773097 5188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:41:11.779346 5188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:41:11.779839 5188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:41:11.781621 5188 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:41:11.784102 5188 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:41:11.784269 5188 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:11.784535 5188 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:41:11.784675 5188 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:11.876116 5188 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.193:38201
I20260810 18:41:11.876189 5380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.193:38201 every 8 connection(s)
I20260810 18:41:11.878002 5188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
I20260810 18:41:11.887526 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 5188
I20260810 18:41:11.888025 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance
I20260810 18:41:11.889370 5011 external_mini_cluster.cc:1428] Setting key 4b931bb116adba21198ccbdfe44b062d
Aug 10 18:41:11 dist-test-slave-7kzw krb5kdc[5021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387271, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:11.914664 5387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:11.882971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:36649 (local address 127.4.228.254:38651)
0810 18:41:11.883973 (+ 1002us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:11.883978 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:11.885495 (+ 1517us) server_negotiation.cc:407] Connection header received
0810 18:41:11.889212 (+ 3717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:11.889219 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:11.889352 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:11.889513 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:11.892356 (+ 2843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:11.893586 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:11.896345 (+ 2759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:11.897017 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:11.897190 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:11.903347 (+ 6157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:11.903353 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:11.903357 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:11.903401 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:11.909718 (+ 6317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:11.910846 (+ 1128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:11.910852 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:11.910854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:11.911058 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:11.911729 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:11.911738 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:11.911743 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:11.912111 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:11.912349 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:11.913615 (+ 1266us) server_negotiation.cc:299] Negotiation successful
0810 18:41:11.914019 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":851,"thread_start_us":542,"threads_started":1}
I20260810 18:41:11.915896 5391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:11.883515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:38651 (local address 127.4.228.193:36649)
0810 18:41:11.885276 (+ 1761us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:11.885334 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:11.888657 (+ 3323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:11.889917 (+ 1260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:11.889927 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:11.891030 (+ 1103us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:11.892075 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:11.892114 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:11.893815 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:11.893819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:11.896161 (+ 2342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:11.896177 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:11.897484 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:11.897495 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:11.897877 (+ 382us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:11.898842 (+ 965us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:41:11.898870 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:11.902885 (+ 4015us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:11.909924 (+ 7039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:11.909941 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:11.909959 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:11.910660 (+ 701us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:11.911210 (+ 550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:11.911215 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:11.911227 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:11.911562 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:11.912275 (+ 713us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:11.912286 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:11.912821 (+ 535us) client_negotiation.cc:769] Sending connection context
0810 18:41:11.913507 (+ 686us) client_negotiation.cc:240] Negotiation successful
0810 18:41:11.914327 (+ 820us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1318,"thread_start_us":911,"threads_started":1}
I20260810 18:41:11.918684 5381 heartbeater.cc:347] Connected to a master server at 127.4.228.254:38651
I20260810 18:41:11.919286 5381 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.4.228.194@KRBTEST.COM; defaulting to no policy
I20260810 18:41:11.920565 5381 heartbeater.cc:511] Master 127.4.228.254:38651 requested a full tablet report, sending...
I20260810 18:41:11.925303 5088 ts_manager.cc:194] Registered new tserver with Master: bcf5ab79a8c74ff399f7ad7da31e709a (127.4.228.193:38201)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.194@KRBTEST.COM" created.
I20260810 18:41:11.928108 5088 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.193@KRBTEST.COM'} at 127.4.228.193:36649
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab.
Entry for principal kudu/127.4.228.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab.
I20260810 18:41:11.958900 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.194:0
--local_ip_for_outbound_sockets=127.4.228.194
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:38651
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:37833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:41:12.605507 5395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:41:12.605984 5395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:41:12.606281 5395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:41:12.618136 5395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:41:12.618345 5395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:41:12.618690 5395 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.194
I20260810 18:41:12.632540 5395 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:37833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:38651
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.194
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:41:12.634202 5395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:41:12.636037 5395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:41:12.658245 5402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:12.658298 5404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:12.659507 5408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:12.660200 5395 server_base.cc:1034] running on GCE node
I20260810 18:41:12.661016 5395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:41:12.662289 5395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:41:12.663643 5395 hybrid_clock.cc:648] HybridClock initialized: now 1786387272663492 us; error 91 us; skew 500 ppm
I20260810 18:41:12.664216 5395 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:41:12 dist-test-slave-7kzw krb5kdc[5021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387272, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:41:12.669642 5395 init.cc:377] Logged in from keytab as kudu/127.4.228.194@KRBTEST.COM (short username kudu)
I20260810 18:41:12.672117 5395 webserver.cc:460] Webserver started at http://127.4.228.194:38063/ using document root <none> and password file <none>
I20260810 18:41:12.673019 5395 fs_manager.cc:359] Metadata directory not provided
I20260810 18:41:12.673240 5395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:41:12.673652 5395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:41:12.677598 5395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-1/data/instance:
uuid: "addddbe219cf47ba986324f33a2f310d"
format_stamp: "Formatted at 2026-08-10 18:41:12 on dist-test-slave-7kzw"
server_key: "0f3c83f76dd26e98756a19e06dd55ead"
server_key_iv: "9543da23a3b8b31ac12aed580b71a60f"
server_key_version: "encryptionkey@0"
I20260810 18:41:12.678668 5395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance:
uuid: "addddbe219cf47ba986324f33a2f310d"
format_stamp: "Formatted at 2026-08-10 18:41:12 on dist-test-slave-7kzw"
server_key: "0f3c83f76dd26e98756a19e06dd55ead"
server_key_iv: "9543da23a3b8b31ac12aed580b71a60f"
server_key_version: "encryptionkey@0"
I20260810 18:41:12.686789 5395 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260810 18:41:12.692664 5416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:12.694808 5395 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.003s
I20260810 18:41:12.695117 5395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-1/wal
uuid: "addddbe219cf47ba986324f33a2f310d"
format_stamp: "Formatted at 2026-08-10 18:41:12 on dist-test-slave-7kzw"
server_key: "0f3c83f76dd26e98756a19e06dd55ead"
server_key_iv: "9543da23a3b8b31ac12aed580b71a60f"
server_key_version: "encryptionkey@0"
I20260810 18:41:12.695524 5395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:41:12.738358 5395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:41:12.745340 5395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:41:12.745806 5395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:41:12.747745 5395 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:41:12.750191 5395 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:41:12.750396 5395 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:12.750723 5395 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:41:12.750876 5395 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:12.840360 5395 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.194:40461
I20260810 18:41:12.840502 5591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.194:40461 every 8 connection(s)
I20260810 18:41:12.842195 5395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
I20260810 18:41:12.852401 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 5395
I20260810 18:41:12.852764 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance
I20260810 18:41:12.853309 5011 external_mini_cluster.cc:1428] Setting key 2516a9dd47f844b25f4033ca47ff7487
Aug 10 18:41:12 dist-test-slave-7kzw krb5kdc[5021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387272, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:12.875662 5594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:12.846413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:54699 (local address 127.4.228.254:38651)
0810 18:41:12.847327 (+ 914us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:12.847333 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:12.849165 (+ 1832us) server_negotiation.cc:407] Connection header received
0810 18:41:12.852549 (+ 3384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:12.852557 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:12.852663 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:12.852949 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:12.855754 (+ 2805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:12.856926 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:12.859433 (+ 2507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:12.860070 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:12.860232 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:12.865911 (+ 5679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:12.865917 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:12.865920 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:12.865958 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:12.870282 (+ 4324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:12.871700 (+ 1418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:12.871704 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:12.871707 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:12.871916 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:12.872670 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:12.872676 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:12.872679 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:12.873062 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:12.873239 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:12.874769 (+ 1530us) server_negotiation.cc:299] Negotiation successful
0810 18:41:12.875118 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":755,"thread_start_us":507,"threads_started":1}
I20260810 18:41:12.877403 5596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:12.847370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:38651 (local address 127.4.228.194:54699)
0810 18:41:12.848862 (+ 1492us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:12.848930 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:12.852146 (+ 3216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:12.853300 (+ 1154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:12.853310 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:12.854410 (+ 1100us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:12.855463 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:12.855496 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:12.857190 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:12.857194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:12.859265 (+ 2071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:12.859279 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:12.860503 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:12.860515 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:12.860890 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:12.861823 (+ 933us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:41:12.861849 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:12.865523 (+ 3674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:12.870514 (+ 4991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:12.870530 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:12.870566 (+ 36us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:12.871520 (+ 954us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:12.872052 (+ 532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:12.872060 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:12.872068 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:12.872486 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:12.873269 (+ 783us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:12.873282 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:12.873947 (+ 665us) client_negotiation.cc:769] Sending connection context
0810 18:41:12.874416 (+ 469us) client_negotiation.cc:240] Negotiation successful
0810 18:41:12.875246 (+ 830us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1209,"thread_start_us":692,"threads_started":1}
I20260810 18:41:12.880142 5592 heartbeater.cc:347] Connected to a master server at 127.4.228.254:38651
I20260810 18:41:12.880622 5592 heartbeater.cc:464] Registering TS with master...
I20260810 18:41:12.882349 5592 heartbeater.cc:511] Master 127.4.228.254:38651 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.4.228.195@KRBTEST.COM; defaulting to no policy
I20260810 18:41:12.884959 5088 ts_manager.cc:194] Registered new tserver with Master: addddbe219cf47ba986324f33a2f310d (127.4.228.194:40461)
I20260810 18:41:12.885994 5088 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.194@KRBTEST.COM'} at 127.4.228.194:54699
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab.
Entry for principal kudu/127.4.228.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab.
I20260810 18:41:12.920397 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.195:0
--local_ip_for_outbound_sockets=127.4.228.195
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:38651
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:37833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:41:12.932479 5381 heartbeater.cc:503] Master 127.4.228.254:38651 was elected leader, sending a full tablet report...
W20260810 18:41:13.564807 5601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:41:13.565263 5601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:41:13.565524 5601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:41:13.575716 5601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:41:13.575994 5601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:41:13.576262 5601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.195
I20260810 18:41:13.591058 5601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:37833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:38651
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.195
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:41:13.592875 5601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:41:13.594710 5601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:41:13.615490 5606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:13.616128 5610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:13.616891 5614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:13.618122 5601 server_base.cc:1034] running on GCE node
I20260810 18:41:13.619015 5601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:41:13.620376 5601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:41:13.621650 5601 hybrid_clock.cc:648] HybridClock initialized: now 1786387273621520 us; error 83 us; skew 500 ppm
I20260810 18:41:13.622130 5601 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:41:13 dist-test-slave-7kzw krb5kdc[5021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387273, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:41:13.626586 5601 init.cc:377] Logged in from keytab as kudu/127.4.228.195@KRBTEST.COM (short username kudu)
I20260810 18:41:13.628859 5601 webserver.cc:460] Webserver started at http://127.4.228.195:36259/ using document root <none> and password file <none>
I20260810 18:41:13.629755 5601 fs_manager.cc:359] Metadata directory not provided
I20260810 18:41:13.630002 5601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:41:13.630456 5601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:41:13.634619 5601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-2/data/instance:
uuid: "64ddeca329634c759de4e1f226fb0223"
format_stamp: "Formatted at 2026-08-10 18:41:13 on dist-test-slave-7kzw"
server_key: "86a33afabd58e91c41bfb5323470ee6e"
server_key_iv: "6a3666f66a4267385539ff21094eeb48"
server_key_version: "encryptionkey@0"
I20260810 18:41:13.635627 5601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance:
uuid: "64ddeca329634c759de4e1f226fb0223"
format_stamp: "Formatted at 2026-08-10 18:41:13 on dist-test-slave-7kzw"
server_key: "86a33afabd58e91c41bfb5323470ee6e"
server_key_iv: "6a3666f66a4267385539ff21094eeb48"
server_key_version: "encryptionkey@0"
I20260810 18:41:13.643707 5601 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260810 18:41:13.649370 5624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:13.651324 5601 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260810 18:41:13.651638 5601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-2/wal
uuid: "64ddeca329634c759de4e1f226fb0223"
format_stamp: "Formatted at 2026-08-10 18:41:13 on dist-test-slave-7kzw"
server_key: "86a33afabd58e91c41bfb5323470ee6e"
server_key_iv: "6a3666f66a4267385539ff21094eeb48"
server_key_version: "encryptionkey@0"
I20260810 18:41:13.651989 5601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:41:13.693535 5601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:41:13.700536 5601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:41:13.701081 5601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:41:13.703065 5601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:41:13.705293 5601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:41:13.705467 5601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:13.705740 5601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:41:13.705879 5601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:13.796299 5601 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.195:43155
I20260810 18:41:13.796428 5796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.195:43155 every 8 connection(s)
I20260810 18:41:13.798237 5601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
I20260810 18:41:13.806921 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 5601
I20260810 18:41:13.807245 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance
I20260810 18:41:13.807896 5011 external_mini_cluster.cc:1428] Setting key ac8910d09772c3366b959f181e5ac444
Aug 10 18:41:13 dist-test-slave-7kzw krb5kdc[5021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387273, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:13.831167 5801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:13.803035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:40707 (local address 127.4.228.254:38651)
0810 18:41:13.804062 (+ 1027us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:13.804068 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:13.805703 (+ 1635us) server_negotiation.cc:407] Connection header received
0810 18:41:13.808856 (+ 3153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:13.808862 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:13.808968 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:13.809194 (+ 226us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:13.811436 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:13.812648 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:13.815540 (+ 2892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:13.816321 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:13.816494 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:13.821783 (+ 5289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:13.821789 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:13.821793 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:13.821833 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:13.826726 (+ 4893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:13.827749 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:13.827758 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:13.827764 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:13.827983 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:13.828608 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:13.828614 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:13.828620 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:13.829067 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:13.829245 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:13.830187 (+ 942us) server_negotiation.cc:299] Negotiation successful
0810 18:41:13.830441 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":885,"thread_start_us":689,"threads_started":1}
I20260810 18:41:13.832511 5802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:13.803741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:38651 (local address 127.4.228.195:40707)
0810 18:41:13.805390 (+ 1649us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:13.805438 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:13.808480 (+ 3042us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:13.809561 (+ 1081us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:13.809571 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:13.810365 (+ 794us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:13.811247 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:13.811268 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:13.812897 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:13.812902 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:13.815336 (+ 2434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:13.815352 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:13.816795 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:13.816806 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:13.817135 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:13.818035 (+ 900us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:41:13.818064 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:13.821374 (+ 3310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:13.826904 (+ 5530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:13.826916 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:13.826944 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:13.827578 (+ 634us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:13.828134 (+ 556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:13.828139 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:13.828144 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:13.828457 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:13.829216 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:13.829226 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:13.829717 (+ 491us) client_negotiation.cc:769] Sending connection context
0810 18:41:13.830087 (+ 370us) client_negotiation.cc:240] Negotiation successful
0810 18:41:13.830888 (+ 801us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1247,"thread_start_us":971,"threads_started":1}
I20260810 18:41:13.834761 5797 heartbeater.cc:347] Connected to a master server at 127.4.228.254:38651
I20260810 18:41:13.835192 5797 heartbeater.cc:464] Registering TS with master...
I20260810 18:41:13.836210 5797 heartbeater.cc:511] Master 127.4.228.254:38651 requested a full tablet report, sending...
I20260810 18:41:13.838402 5088 ts_manager.cc:194] Registered new tserver with Master: 64ddeca329634c759de4e1f226fb0223 (127.4.228.195:43155)
I20260810 18:41:13.839985 5088 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.195@KRBTEST.COM'} at 127.4.228.195:40707
I20260810 18:41:13.840870 5011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 18:41:13 dist-test-slave-7kzw krb5kdc[5021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387273, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
I20260810 18:41:13.889611 5592 heartbeater.cc:503] Master 127.4.228.254:38651 was elected leader, sending a full tablet report...
Aug 10 18:41:13 dist-test-slave-7kzw krb5kdc[5021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387273, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.228.193@KRBTEST.COM
I20260810 18:41:13.917449 5809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:13.891480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43320 (local address 127.4.228.193:38201)
0810 18:41:13.892789 (+ 1309us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:13.892801 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:13.892849 (+ 48us) server_negotiation.cc:407] Connection header received
0810 18:41:13.893024 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:13.893043 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:13.893344 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:13.893506 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:13.895803 (+ 2297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:13.897477 (+ 1674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:13.899966 (+ 2489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:13.901185 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:13.901318 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:13.908302 (+ 6984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:13.908320 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:13.908341 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:13.908409 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:13.912910 (+ 4501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:13.914070 (+ 1160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:13.914080 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:13.914082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:13.914341 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:13.915009 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:13.915014 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:13.915017 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:13.915566 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:13.915967 (+ 401us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:13.916361 (+ 394us) server_negotiation.cc:299] Negotiation successful
0810 18:41:13.916714 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1030,"thread_start_us":1076,"threads_started":1}
W20260810 18:41:13.919648 5248 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43320
Aug 10 18:41:13 dist-test-slave-7kzw krb5kdc[5021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 18:41:13 dist-test-slave-7kzw krb5kdc[5021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.193@KRBTEST.COM
I20260810 18:41:13.986714 5809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:13.966717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43326 (local address 127.4.228.193:38201)
0810 18:41:13.966921 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:13.966927 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:13.966950 (+ 23us) server_negotiation.cc:407] Connection header received
0810 18:41:13.967042 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:13.967046 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:13.967114 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:13.967208 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:13.969169 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:13.969921 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:13.971859 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:13.972314 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:13.972424 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:13.978930 (+ 6506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:13.978937 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:13.978941 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:13.978993 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:13.983244 (+ 4251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:13.984063 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:13.984069 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:13.984072 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:13.984370 (+ 298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:13.984811 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:13.984815 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:13.984817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:13.985243 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:13.985509 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:13.985950 (+ 441us) server_negotiation.cc:299] Negotiation successful
0810 18:41:13.986218 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 18:41:13.990136 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 5188
I20260810 18:41:14.011338 5809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:13.991616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43334 (local address 127.4.228.193:38201)
0810 18:41:13.991932 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:13.991940 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:13.992288 (+ 348us) server_negotiation.cc:407] Connection header received
0810 18:41:13.992528 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:13.992532 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:13.992601 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:13.992705 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:13.994763 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:13.995992 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:13.998027 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:13.998963 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:13.999187 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:14.002815 (+ 3628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:14.002822 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:14.002826 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:14.002874 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:14.007160 (+ 4286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:14.008234 (+ 1074us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:14.008239 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:14.008242 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:14.008525 (+ 283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:14.009221 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:14.009226 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:14.009228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:14.009690 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:14.009947 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:14.010371 (+ 424us) server_negotiation.cc:299] Negotiation successful
0810 18:41:14.010710 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 18:41:14.012844 5371 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:41:14.111522 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 5188
I20260810 18:41:14.142695 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 5395
Aug 10 18:41:14 dist-test-slave-7kzw krb5kdc[5021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.194@KRBTEST.COM
I20260810 18:41:14.163915 5819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:14.143903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56694 (local address 127.4.228.194:40461)
0810 18:41:14.144987 (+ 1084us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:14.144999 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:14.145034 (+ 35us) server_negotiation.cc:407] Connection header received
0810 18:41:14.145115 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:14.145124 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:14.145490 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:14.145696 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:14.147584 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:14.148933 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:14.150539 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:14.151212 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:14.151343 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:14.156987 (+ 5644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:14.157008 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:14.157032 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:14.157143 (+ 111us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:14.160372 (+ 3229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:14.161258 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:14.161267 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:14.161270 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:14.161431 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:14.161901 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:14.161905 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:14.161908 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:14.162335 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:14.162719 (+ 384us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:14.163185 (+ 466us) server_negotiation.cc:299] Negotiation successful
0810 18:41:14.163474 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":823,"thread_start_us":564,"threads_started":1}
I20260810 18:41:14.165345 5585 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:41:14.253705 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 5395
I20260810 18:41:14.284571 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 5601
Aug 10 18:41:14 dist-test-slave-7kzw krb5kdc[5021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.195@KRBTEST.COM
I20260810 18:41:14.304028 5821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:14.285686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44724 (local address 127.4.228.195:43155)
0810 18:41:14.286263 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:14.286271 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:14.286303 (+ 32us) server_negotiation.cc:407] Connection header received
0810 18:41:14.286369 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:14.286376 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:14.286692 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:14.286843 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:14.288312 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:14.289460 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:14.290981 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:14.291638 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:14.291790 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:14.296997 (+ 5207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:14.297016 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:14.297037 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:14.297087 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:14.300708 (+ 3621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:14.301587 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:14.301595 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:14.301598 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:14.301791 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:14.302179 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:14.302183 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:14.302186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:14.302840 (+ 654us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:14.303101 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:14.303423 (+ 322us) server_negotiation.cc:299] Negotiation successful
0810 18:41:14.303686 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":264,"threads_started":1}
I20260810 18:41:14.305519 5791 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:41:14.395287 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 5601
I20260810 18:41:14.424721 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 5040
I20260810 18:41:14.426267 5135 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:41:14.533828 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 5040
2026-08-10T18:41:14Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (6527 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:41:14 dist-test-slave-7kzw krb5kdc[5829](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:41:14 dist-test-slave-7kzw krb5kdc[5829](info): set up 2 sockets
Aug 10 18:41:14 dist-test-slave-7kzw krb5kdc[5829](info): commencing operation
krb5kdc: starting...
W20260810 18:41:15.874450 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.282s user 0.004s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:41:16 dist-test-slave-7kzw krb5kdc[5829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:41:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:41:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.228.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
I20260810 18:41:16.116709 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:37085
--webserver_interface=127.4.228.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:43803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:37085
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:41:16.474563 5846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:41:16.474908 5846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:41:16.475080 5846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:41:16.482581 5846 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:41:16.482697 5846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:41:16.482800 5846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:41:16.482908 5846 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:41:16.482972 5846 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:41:16.493309 5846 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:43803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:37085
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:37085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:41:16.494930 5846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:41:16.496376 5846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:41:16.510156 5853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:16.510303 5859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:16.510315 5854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:16.511418 5846 server_base.cc:1034] running on GCE node
I20260810 18:41:16.512709 5846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:41:16.514581 5846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:41:16.515929 5846 hybrid_clock.cc:648] HybridClock initialized: now 1786387276515860 us; error 58 us; skew 500 ppm
I20260810 18:41:16.516396 5846 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:41:16 dist-test-slave-7kzw krb5kdc[5829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387276, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:41:16.520296 5846 init.cc:377] Logged in from keytab as kudu/127.4.228.254@KRBTEST.COM (short username kudu)
I20260810 18:41:16.522091 5846 webserver.cc:460] Webserver started at http://127.4.228.254:34079/ using document root <none> and password file <none>
I20260810 18:41:16.522995 5846 fs_manager.cc:359] Metadata directory not provided
I20260810 18:41:16.523118 5846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:41:16.523437 5846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:41:16.527367 5846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "a8034241247344c0b1a585fcb6b5bc87"
format_stamp: "Formatted at 2026-08-10 18:41:16 on dist-test-slave-7kzw"
server_key: "2107792b0581f4ee3c6b26830d947449"
server_key_iv: "3d59ad34405b063dc08514c2ca45ef53"
server_key_version: "encryptionkey@0"
I20260810 18:41:16.528227 5846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "a8034241247344c0b1a585fcb6b5bc87"
format_stamp: "Formatted at 2026-08-10 18:41:16 on dist-test-slave-7kzw"
server_key: "2107792b0581f4ee3c6b26830d947449"
server_key_iv: "3d59ad34405b063dc08514c2ca45ef53"
server_key_version: "encryptionkey@0"
I20260810 18:41:16.534574 5846 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260810 18:41:16.539006 5868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:16.540781 5846 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260810 18:41:16.541016 5846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "a8034241247344c0b1a585fcb6b5bc87"
format_stamp: "Formatted at 2026-08-10 18:41:16 on dist-test-slave-7kzw"
server_key: "2107792b0581f4ee3c6b26830d947449"
server_key_iv: "3d59ad34405b063dc08514c2ca45ef53"
server_key_version: "encryptionkey@0"
I20260810 18:41:16.541271 5846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:41:16.569219 5846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:41:16.575519 5846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:41:16.575878 5846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:41:16.597546 5846 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:37085
I20260810 18:41:16.597573 5951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:37085 every 8 connection(s)
I20260810 18:41:16.599354 5846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:41:16.601691 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 5846
I20260810 18:41:16.601984 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:41:16.602785 5011 external_mini_cluster.cc:1428] Setting key 0b2d53012fabdec416410ca927be5e63
I20260810 18:41:16.604568 5952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 18:41:16 dist-test-slave-7kzw krb5kdc[5829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:16.620034 5952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8034241247344c0b1a585fcb6b5bc87: Bootstrap starting.
I20260810 18:41:16.623839 5952 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8034241247344c0b1a585fcb6b5bc87: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:16.625120 5952 log.cc:824] T 00000000000000000000000000000000 P a8034241247344c0b1a585fcb6b5bc87: Log is configured to *not* fsync() on all Append() calls
I20260810 18:41:16.628968 5952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8034241247344c0b1a585fcb6b5bc87: No bootstrap required, opened a new log
I20260810 18:41:16.629607 5955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:16.605146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33192 (local address 127.4.228.254:37085)
0810 18:41:16.606101 (+ 955us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:16.606122 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:16.606182 (+ 60us) server_negotiation.cc:407] Connection header received
0810 18:41:16.608519 (+ 2337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:16.608556 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:16.609211 (+ 655us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:16.609789 (+ 578us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:16.611490 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:16.613707 (+ 2217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:16.615219 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:16.616330 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:16.616537 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:16.621190 (+ 4653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:16.621200 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:16.621217 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:16.621263 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:16.624674 (+ 3411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:16.625632 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:16.625640 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:16.625648 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:16.625956 (+ 308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:16.626407 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:16.626416 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:16.626422 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:16.627153 (+ 731us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:16.627416 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:16.628022 (+ 606us) server_negotiation.cc:299] Negotiation successful
0810 18:41:16.628470 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":373,"threads_started":1}
I20260810 18:41:16.637501 5952 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8034241247344c0b1a585fcb6b5bc87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8034241247344c0b1a585fcb6b5bc87" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 37085 } }
I20260810 18:41:16.638010 5952 raft_consensus.cc:374] T 00000000000000000000000000000000 P a8034241247344c0b1a585fcb6b5bc87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:16.638123 5952 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8034241247344c0b1a585fcb6b5bc87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8034241247344c0b1a585fcb6b5bc87, State: Initialized, Role: FOLLOWER
I20260810 18:41:16.638856 5952 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8034241247344c0b1a585fcb6b5bc87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8034241247344c0b1a585fcb6b5bc87" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 37085 } }
I20260810 18:41:16.639088 5952 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8034241247344c0b1a585fcb6b5bc87 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:41:16.639257 5952 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8034241247344c0b1a585fcb6b5bc87 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:41:16.639472 5952 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8034241247344c0b1a585fcb6b5bc87 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:16.642248 5952 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8034241247344c0b1a585fcb6b5bc87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8034241247344c0b1a585fcb6b5bc87" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 37085 } }
I20260810 18:41:16.642984 5952 leader_election.cc:302] T 00000000000000000000000000000000 P a8034241247344c0b1a585fcb6b5bc87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8034241247344c0b1a585fcb6b5bc87; no voters:
I20260810 18:41:16.643723 5952 leader_election.cc:288] T 00000000000000000000000000000000 P a8034241247344c0b1a585fcb6b5bc87 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:41:16.644002 5957 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8034241247344c0b1a585fcb6b5bc87 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:41:16.644785 5957 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8034241247344c0b1a585fcb6b5bc87 [term 1 LEADER]: Becoming Leader. State: Replica: a8034241247344c0b1a585fcb6b5bc87, State: Running, Role: LEADER
I20260810 18:41:16.645419 5957 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8034241247344c0b1a585fcb6b5bc87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8034241247344c0b1a585fcb6b5bc87" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 37085 } }
I20260810 18:41:16.646252 5952 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8034241247344c0b1a585fcb6b5bc87 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:41:16.648741 5959 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8034241247344c0b1a585fcb6b5bc87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a8034241247344c0b1a585fcb6b5bc87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8034241247344c0b1a585fcb6b5bc87" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 37085 } } }
I20260810 18:41:16.648877 5960 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8034241247344c0b1a585fcb6b5bc87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8034241247344c0b1a585fcb6b5bc87. Latest consensus state: current_term: 1 leader_uuid: "a8034241247344c0b1a585fcb6b5bc87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8034241247344c0b1a585fcb6b5bc87" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 37085 } } }
I20260810 18:41:16.649178 5959 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8034241247344c0b1a585fcb6b5bc87 [sys.catalog]: This master's current role is: LEADER
I20260810 18:41:16.649328 5960 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8034241247344c0b1a585fcb6b5bc87 [sys.catalog]: This master's current role is: LEADER
I20260810 18:41:16.650249 5966 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:41:16.656569 5966 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:41:16.669081 5966 catalog_manager.cc:1347] Generated new cluster ID: ce4c129876d2477dba3a0ce4f0dba628
I20260810 18:41:16.669224 5966 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:41:16.679965 5966 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:41:16.681442 5966 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:41:16.687986 5966 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8034241247344c0b1a585fcb6b5bc87: Generated new TSK 0
I20260810 18:41:16.688918 5966 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.228.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab.
Entry for principal kudu/127.4.228.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab.
I20260810 18:41:16.758978 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.193:0
--local_ip_for_outbound_sockets=127.4.228.193
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:37085
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:43803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:41:17.175163 5983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:41:17.175508 5983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:41:17.175611 5983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:41:17.183085 5983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:41:17.183218 5983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:41:17.183341 5983 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.193
I20260810 18:41:17.194722 5983 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:43803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:37085
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.193
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:41:17.196221 5983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:41:17.197749 5983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:41:17.213297 5991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:17.213349 5993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:17.213318 5997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:17.213850 5983 server_base.cc:1034] running on GCE node
I20260810 18:41:17.214881 5983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:41:17.215945 5983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:41:17.217257 5983 hybrid_clock.cc:648] HybridClock initialized: now 1786387277217208 us; error 38 us; skew 500 ppm
I20260810 18:41:17.217653 5983 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:41:17 dist-test-slave-7kzw krb5kdc[5829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387277, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:41:17.221340 5983 init.cc:377] Logged in from keytab as kudu/127.4.228.193@KRBTEST.COM (short username kudu)
I20260810 18:41:17.223095 5983 webserver.cc:460] Webserver started at http://127.4.228.193:44407/ using document root <none> and password file <none>
I20260810 18:41:17.223879 5983 fs_manager.cc:359] Metadata directory not provided
I20260810 18:41:17.224037 5983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:41:17.224388 5983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:41:17.227712 5983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-0/data/instance:
uuid: "b468cba8f1d041a88f09a09beba8d713"
format_stamp: "Formatted at 2026-08-10 18:41:17 on dist-test-slave-7kzw"
server_key: "97f14bf64b7818569c2ce45c6e82ad5d"
server_key_iv: "38199ce93d5073b55e602b1272ae11b7"
server_key_version: "encryptionkey@0"
I20260810 18:41:17.228554 5983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance:
uuid: "b468cba8f1d041a88f09a09beba8d713"
format_stamp: "Formatted at 2026-08-10 18:41:17 on dist-test-slave-7kzw"
server_key: "97f14bf64b7818569c2ce45c6e82ad5d"
server_key_iv: "38199ce93d5073b55e602b1272ae11b7"
server_key_version: "encryptionkey@0"
I20260810 18:41:17.234162 5983 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 18:41:17.238322 6006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:17.240135 5983 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 18:41:17.240360 5983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-0/wal
uuid: "b468cba8f1d041a88f09a09beba8d713"
format_stamp: "Formatted at 2026-08-10 18:41:17 on dist-test-slave-7kzw"
server_key: "97f14bf64b7818569c2ce45c6e82ad5d"
server_key_iv: "38199ce93d5073b55e602b1272ae11b7"
server_key_version: "encryptionkey@0"
I20260810 18:41:17.240600 5983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:41:17.264763 5983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:41:17.269975 5983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:41:17.270318 5983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:41:17.271486 5983 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:41:17.273032 5983 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:41:17.273149 5983 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:17.273312 5983 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:41:17.273388 5983 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:17.310702 5983 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.193:45777
I20260810 18:41:17.310803 6169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.193:45777 every 8 connection(s)
I20260810 18:41:17.312146 5983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
I20260810 18:41:17.319389 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 5983
I20260810 18:41:17.319660 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance
I20260810 18:41:17.320312 5011 external_mini_cluster.cc:1428] Setting key bddb61dc6152327cb606ce7644a88777
Aug 10 18:41:17 dist-test-slave-7kzw krb5kdc[5829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387277, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:17.337695 6174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:17.315084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:47695 (local address 127.4.228.254:37085)
0810 18:41:17.315620 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:17.315631 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:17.316691 (+ 1060us) server_negotiation.cc:407] Connection header received
0810 18:41:17.318969 (+ 2278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:17.318977 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:17.319087 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:17.319228 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:17.321163 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:17.322201 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:17.325057 (+ 2856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:17.325454 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:17.325592 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:17.330804 (+ 5212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:17.330814 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:17.330822 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:17.330876 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:17.333900 (+ 3024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:17.334973 (+ 1073us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:17.334977 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:17.334980 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:17.335137 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:17.335763 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:17.335768 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:17.335770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:17.336081 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:17.336234 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:17.337197 (+ 963us) server_negotiation.cc:299] Negotiation successful
0810 18:41:17.337394 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":268,"threads_started":1}
I20260810 18:41:17.339437 6175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:17.315685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:37085 (local address 127.4.228.193:47695)
0810 18:41:17.316493 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:17.316541 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:17.318650 (+ 2109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:17.319525 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:17.319541 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:17.320244 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:17.320951 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:17.320970 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:17.322409 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:17.322413 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:17.324860 (+ 2447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:17.324881 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:17.325739 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:17.325752 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:17.326146 (+ 394us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:17.327093 (+ 947us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:41:17.327142 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:17.330521 (+ 3379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:17.334071 (+ 3550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:17.334083 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:17.334106 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:17.334834 (+ 728us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:17.335298 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:17.335306 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:17.335312 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:17.335613 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:17.336227 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:17.336237 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:17.336741 (+ 504us) client_negotiation.cc:769] Sending connection context
0810 18:41:17.337067 (+ 326us) client_negotiation.cc:240] Negotiation successful
0810 18:41:17.337757 (+ 690us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":288,"threads_started":1}
I20260810 18:41:17.341311 6171 heartbeater.cc:347] Connected to a master server at 127.4.228.254:37085
I20260810 18:41:17.341673 6171 heartbeater.cc:464] Registering TS with master...
I20260810 18:41:17.342595 6171 heartbeater.cc:511] Master 127.4.228.254:37085 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.4.228.194@KRBTEST.COM; defaulting to no policy
I20260810 18:41:17.345363 5892 ts_manager.cc:194] Registered new tserver with Master: b468cba8f1d041a88f09a09beba8d713 (127.4.228.193:45777)
I20260810 18:41:17.347508 5892 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.193@KRBTEST.COM'} at 127.4.228.193:47695
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab.
Entry for principal kudu/127.4.228.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab.
I20260810 18:41:17.376346 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.194:0
--local_ip_for_outbound_sockets=127.4.228.194
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:37085
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:43803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:41:17.741814 6181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:41:17.742146 6181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:41:17.742322 6181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:41:17.750597 6181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:41:17.750730 6181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:41:17.750911 6181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.194
I20260810 18:41:17.761444 6181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:43803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:37085
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.194
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:41:17.763069 6181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:41:17.764652 6181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:41:17.779068 6189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:17.779181 6190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:17.779215 6192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:17.780179 6181 server_base.cc:1034] running on GCE node
I20260810 18:41:17.780941 6181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:41:17.782004 6181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:41:17.783275 6181 hybrid_clock.cc:648] HybridClock initialized: now 1786387277783203 us; error 63 us; skew 500 ppm
I20260810 18:41:17.783663 6181 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:41:17 dist-test-slave-7kzw krb5kdc[5829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387277, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:41:17.791834 6181 init.cc:377] Logged in from keytab as kudu/127.4.228.194@KRBTEST.COM (short username kudu)
I20260810 18:41:17.793605 6181 webserver.cc:460] Webserver started at http://127.4.228.194:40201/ using document root <none> and password file <none>
I20260810 18:41:17.794354 6181 fs_manager.cc:359] Metadata directory not provided
I20260810 18:41:17.794512 6181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:41:17.794847 6181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:41:17.798017 6181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-1/data/instance:
uuid: "c7e07a5c706a46e7b64c2365347d8dea"
format_stamp: "Formatted at 2026-08-10 18:41:17 on dist-test-slave-7kzw"
server_key: "d4472fbafe347d27c33786cb1c795f81"
server_key_iv: "6179f7250df6cf9448b0016ec263509d"
server_key_version: "encryptionkey@0"
I20260810 18:41:17.798964 6181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance:
uuid: "c7e07a5c706a46e7b64c2365347d8dea"
format_stamp: "Formatted at 2026-08-10 18:41:17 on dist-test-slave-7kzw"
server_key: "d4472fbafe347d27c33786cb1c795f81"
server_key_iv: "6179f7250df6cf9448b0016ec263509d"
server_key_version: "encryptionkey@0"
I20260810 18:41:17.805460 6181 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.003s
I20260810 18:41:17.810187 6201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:17.812116 6181 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 18:41:17.812341 6181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-1/wal
uuid: "c7e07a5c706a46e7b64c2365347d8dea"
format_stamp: "Formatted at 2026-08-10 18:41:17 on dist-test-slave-7kzw"
server_key: "d4472fbafe347d27c33786cb1c795f81"
server_key_iv: "6179f7250df6cf9448b0016ec263509d"
server_key_version: "encryptionkey@0"
I20260810 18:41:17.812583 6181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:41:17.847268 6181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:41:17.852867 6181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:41:17.853250 6181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:41:17.854516 6181 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:41:17.856236 6181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:41:17.856324 6181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:17.856526 6181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:41:17.856587 6181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:17.894428 6181 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.194:40531
I20260810 18:41:17.894495 6360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.194:40531 every 8 connection(s)
I20260810 18:41:17.895953 6181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
I20260810 18:41:17.901084 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 6181
I20260810 18:41:17.901292 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance
I20260810 18:41:17.901774 5011 external_mini_cluster.cc:1428] Setting key fe6d0590d41e570de91dace1365375ab
Aug 10 18:41:17 dist-test-slave-7kzw krb5kdc[5829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387277, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:17.921015 6370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:17.899065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:60113 (local address 127.4.228.254:37085)
0810 18:41:17.899633 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:17.899641 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:17.900593 (+ 952us) server_negotiation.cc:407] Connection header received
0810 18:41:17.903092 (+ 2499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:17.903106 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:17.903206 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:17.903382 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:17.905430 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:17.906928 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:17.908854 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:17.909725 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:17.909901 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:17.913774 (+ 3873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:17.913780 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:17.913783 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:17.913820 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:17.917108 (+ 3288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:17.918244 (+ 1136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:17.918249 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:17.918252 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:17.918410 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:17.919037 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:17.919040 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:17.919043 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:17.919382 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:17.919524 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:17.920487 (+ 963us) server_negotiation.cc:299] Negotiation successful
0810 18:41:17.920689 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":411,"thread_start_us":259,"threads_started":1}
I20260810 18:41:17.922987 6372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:17.899516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:37085 (local address 127.4.228.194:60113)
0810 18:41:17.900398 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:17.900453 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:17.902688 (+ 2235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:17.903678 (+ 990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:17.903687 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:17.904511 (+ 824us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:17.905218 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:17.905239 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:17.907220 (+ 1981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:17.907228 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:17.908663 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:17.908678 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:17.909997 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:17.910002 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:17.910226 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:17.911132 (+ 906us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:41:17.911164 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:17.913491 (+ 2327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:17.917286 (+ 3795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:17.917324 (+ 38us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:17.917349 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:17.918098 (+ 749us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:17.918612 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:17.918620 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:17.918626 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:17.918913 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:17.919547 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:17.919558 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:17.920074 (+ 516us) client_negotiation.cc:769] Sending connection context
0810 18:41:17.920413 (+ 339us) client_negotiation.cc:240] Negotiation successful
0810 18:41:17.921075 (+ 662us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":331,"threads_started":1}
I20260810 18:41:17.924973 6362 heartbeater.cc:347] Connected to a master server at 127.4.228.254:37085
I20260810 18:41:17.925372 6362 heartbeater.cc:464] Registering TS with master...
I20260810 18:41:17.926265 6362 heartbeater.cc:511] Master 127.4.228.254:37085 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.4.228.195@KRBTEST.COM; defaulting to no policy
I20260810 18:41:17.928399 5892 ts_manager.cc:194] Registered new tserver with Master: c7e07a5c706a46e7b64c2365347d8dea (127.4.228.194:40531)
I20260810 18:41:17.929536 5892 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.194@KRBTEST.COM'} at 127.4.228.194:60113
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab.
Entry for principal kudu/127.4.228.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab.
I20260810 18:41:17.962153 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.195:0
--local_ip_for_outbound_sockets=127.4.228.195
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:37085
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:43803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:41:18.320454 6380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:41:18.320768 6380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:41:18.320866 6380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:41:18.328900 6380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:41:18.329031 6380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:41:18.329151 6380 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.195
I20260810 18:41:18.339591 6380 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:43803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:37085
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.195
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:41:18.341019 6380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:41:18.342398 6380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 18:41:18.351227 6171 heartbeater.cc:503] Master 127.4.228.254:37085 was elected leader, sending a full tablet report...
W20260810 18:41:18.357349 6386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:18.357322 6388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:18.357404 6385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:18.358354 6380 server_base.cc:1034] running on GCE node
I20260810 18:41:18.359112 6380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:41:18.360157 6380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:41:18.361377 6380 hybrid_clock.cc:648] HybridClock initialized: now 1786387278361318 us; error 42 us; skew 500 ppm
I20260810 18:41:18.361696 6380 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:41:18 dist-test-slave-7kzw krb5kdc[5829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387278, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:41:18.365571 6380 init.cc:377] Logged in from keytab as kudu/127.4.228.195@KRBTEST.COM (short username kudu)
I20260810 18:41:18.367435 6380 webserver.cc:460] Webserver started at http://127.4.228.195:41875/ using document root <none> and password file <none>
I20260810 18:41:18.368242 6380 fs_manager.cc:359] Metadata directory not provided
I20260810 18:41:18.368393 6380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:41:18.368788 6380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:41:18.372196 6380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-2/data/instance:
uuid: "7279f4ac4a7d46439ff82fc1a11babfa"
format_stamp: "Formatted at 2026-08-10 18:41:18 on dist-test-slave-7kzw"
server_key: "24c680702870066d1d4d6cf140a51604"
server_key_iv: "a32ae788a22f4a76cc8f068878d62c4c"
server_key_version: "encryptionkey@0"
I20260810 18:41:18.373168 6380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance:
uuid: "7279f4ac4a7d46439ff82fc1a11babfa"
format_stamp: "Formatted at 2026-08-10 18:41:18 on dist-test-slave-7kzw"
server_key: "24c680702870066d1d4d6cf140a51604"
server_key_iv: "a32ae788a22f4a76cc8f068878d62c4c"
server_key_version: "encryptionkey@0"
I20260810 18:41:18.379367 6380 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260810 18:41:18.383529 6396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:18.385169 6380 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:41:18.385361 6380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-2/wal
uuid: "7279f4ac4a7d46439ff82fc1a11babfa"
format_stamp: "Formatted at 2026-08-10 18:41:18 on dist-test-slave-7kzw"
server_key: "24c680702870066d1d4d6cf140a51604"
server_key_iv: "a32ae788a22f4a76cc8f068878d62c4c"
server_key_version: "encryptionkey@0"
I20260810 18:41:18.385576 6380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:41:18.401850 6380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:41:18.407300 6380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:41:18.407625 6380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:41:18.408738 6380 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:41:18.410223 6380 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:41:18.410342 6380 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:18.410526 6380 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:41:18.410636 6380 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260810 18:41:18.452744 6380 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.195:33401
I20260810 18:41:18.452872 6556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.195:33401 every 8 connection(s)
I20260810 18:41:18.454545 6380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
I20260810 18:41:18.457072 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 6380
I20260810 18:41:18.457367 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance
I20260810 18:41:18.458009 5011 external_mini_cluster.cc:1428] Setting key 0eecaa5a025a2c47376746db6a8f3c2e
Aug 10 18:41:18 dist-test-slave-7kzw krb5kdc[5829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387278, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:18.485957 6559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:18.458754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:37949 (local address 127.4.228.254:37085)
0810 18:41:18.459980 (+ 1226us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:18.459985 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:18.461905 (+ 1920us) server_negotiation.cc:407] Connection header received
0810 18:41:18.464923 (+ 3018us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:18.464927 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:18.465002 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:18.465115 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:18.467442 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:18.468514 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:18.470624 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:18.471335 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:18.471455 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:18.477504 (+ 6049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:18.477510 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:18.477514 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:18.477555 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:18.481880 (+ 4325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:18.483043 (+ 1163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:18.483048 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:18.483051 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:18.483255 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:18.483848 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:18.483852 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:18.483854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:18.484242 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:18.484407 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:18.485301 (+ 894us) server_negotiation.cc:299] Negotiation successful
0810 18:41:18.485497 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1046,"thread_start_us":757,"threads_started":1}
I20260810 18:41:18.487457 6560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:18.459892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:37085 (local address 127.4.228.195:37949)
0810 18:41:18.461695 (+ 1803us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:18.461759 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:18.464634 (+ 2875us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:18.465443 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:18.465453 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:18.466205 (+ 752us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:18.467200 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:18.467222 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:18.468714 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:18.468719 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:18.470388 (+ 1669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:18.470402 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:18.471649 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:18.471661 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:18.471971 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:18.472869 (+ 898us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:41:18.472896 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:18.477179 (+ 4283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:18.482087 (+ 4908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:18.482097 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:18.482115 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:18.482879 (+ 764us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:18.483373 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:18.483377 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:18.483379 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:18.483719 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:18.484399 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:18.484407 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:18.484891 (+ 484us) client_negotiation.cc:769] Sending connection context
0810 18:41:18.485153 (+ 262us) client_negotiation.cc:240] Negotiation successful
0810 18:41:18.485672 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1303,"thread_start_us":966,"threads_started":1}
I20260810 18:41:18.489436 6557 heartbeater.cc:347] Connected to a master server at 127.4.228.254:37085
I20260810 18:41:18.489856 6557 heartbeater.cc:464] Registering TS with master...
I20260810 18:41:18.491328 6557 heartbeater.cc:511] Master 127.4.228.254:37085 requested a full tablet report, sending...
I20260810 18:41:18.493280 5892 ts_manager.cc:194] Registered new tserver with Master: 7279f4ac4a7d46439ff82fc1a11babfa (127.4.228.195:33401)
I20260810 18:41:18.494550 5892 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.195@KRBTEST.COM'} at 127.4.228.195:37949
I20260810 18:41:18.502866 5011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:41:18.532212 6559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:18.512712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33208 (local address 127.4.228.254:37085)
0810 18:41:18.512906 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:18.512911 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:18.514046 (+ 1135us) server_negotiation.cc:407] Connection header received
0810 18:41:18.514284 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:18.514289 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:18.514362 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:18.514507 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:18.517704 (+ 3197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:18.518752 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:18.520469 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:18.520992 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:18.521196 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:18.524541 (+ 3345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:18.524547 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:18.524550 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:18.524590 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:18.528562 (+ 3972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:18.529464 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:18.529470 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:18.529481 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:18.529654 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:18.530134 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:18.530140 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:18.530146 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:18.530672 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:18.531033 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:18.531418 (+ 385us) server_negotiation.cc:299] Negotiation successful
0810 18:41:18.531724 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 18:41:18.542968 5892 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33208:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:41:18.554507 5892 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:41:18.598131 6580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:18.580509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51206 (local address 127.4.228.193:45777)
0810 18:41:18.581599 (+ 1090us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:18.581608 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:18.585248 (+ 3640us) server_negotiation.cc:407] Connection header received
0810 18:41:18.585646 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:18.585657 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:18.585971 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:18.586194 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:41:18.587708 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:18.589727 (+ 2019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:18.593751 (+ 4024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:18.595832 (+ 2081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:18.596014 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:18.596320 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:18.597327 (+ 1007us) server_negotiation.cc:299] Negotiation successful
0810 18:41:18.597568 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":869,"thread_start_us":994,"threads_started":1}
I20260810 18:41:18.598273 6575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:18.577488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54456 (local address 127.4.228.194:40531)
0810 18:41:18.578958 (+ 1470us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:18.578968 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:18.580435 (+ 1467us) server_negotiation.cc:407] Connection header received
0810 18:41:18.581631 (+ 1196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:18.581644 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:18.582000 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:18.582341 (+ 341us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:41:18.585706 (+ 3365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:18.587771 (+ 2065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:18.592584 (+ 4813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:18.594977 (+ 2393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:18.595123 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:18.595576 (+ 453us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:18.597468 (+ 1892us) server_negotiation.cc:299] Negotiation successful
0810 18:41:18.597712 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1257,"thread_start_us":756,"threads_started":1}
I20260810 18:41:18.600010 6574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:18.577152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:40531 (local address 127.0.0.1:54456)
0810 18:41:18.580120 (+ 2968us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:18.580178 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:18.581165 (+ 987us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:18.583022 (+ 1857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:18.583034 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:18.583092 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:41:18.585211 (+ 2119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:18.585227 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:18.587985 (+ 2758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:18.587993 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:18.592290 (+ 4297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:18.592310 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:18.595902 (+ 3592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:18.595919 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:18.596286 (+ 367us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:18.596373 (+ 87us) client_negotiation.cc:769] Sending connection context
0810 18:41:18.596529 (+ 156us) client_negotiation.cc:240] Negotiation successful
0810 18:41:18.596994 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2529,"mutex_wait_us":24,"thread_start_us":2119,"threads_started":1}
I20260810 18:41:18.601116 6579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:18.578829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55538 (local address 127.4.228.195:33401)
0810 18:41:18.584331 (+ 5502us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:18.584355 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:18.584424 (+ 69us) server_negotiation.cc:407] Connection header received
0810 18:41:18.584554 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:18.584564 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:18.584836 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:18.585268 (+ 432us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:41:18.587140 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:18.589715 (+ 2575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:18.594555 (+ 4840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:18.597216 (+ 2661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:18.597387 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:18.597636 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:18.600168 (+ 2532us) server_negotiation.cc:299] Negotiation successful
0810 18:41:18.600445 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5090,"thread_start_us":4728,"threads_started":1}
I20260810 18:41:18.605160 6582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:18.580233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:45777 (local address 127.0.0.1:51206)
0810 18:41:18.585019 (+ 4786us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:18.585050 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:18.585364 (+ 314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:18.586186 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:18.586191 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:18.586222 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:41:18.587398 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:18.587415 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:18.589936 (+ 2521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:18.589942 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:18.593518 (+ 3576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:18.593537 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:18.596544 (+ 3007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:18.596553 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:18.596803 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:18.596837 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 18:41:18.596937 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 18:41:18.597104 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4378,"mutex_wait_us":65,"thread_start_us":4345,"threads_started":1}
I20260810 18:41:18.605249 6577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:18.578321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:33401 (local address 127.0.0.1:55538)
0810 18:41:18.583257 (+ 4936us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:18.583283 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:18.584044 (+ 761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:18.585092 (+ 1048us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:18.585096 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:18.585126 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:41:18.586687 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:18.586705 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:18.589928 (+ 3223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:18.589933 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:18.594249 (+ 4316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:18.594266 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:18.599359 (+ 5093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:18.599372 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:18.599688 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:18.599747 (+ 59us) client_negotiation.cc:769] Sending connection context
0810 18:41:18.599920 (+ 173us) client_negotiation.cc:240] Negotiation successful
0810 18:41:18.600111 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4674,"mutex_wait_us":26,"thread_start_us":3009,"threads_started":1}
I20260810 18:41:18.607100 6277 tablet_service.cc:1467] Processing CreateTablet for tablet 4cbbbecf96b646009403a0c091c8a27b (DEFAULT_TABLE table=test-table [id=f5611bcd2ed44cb49094c6231b89a89c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:41:18.609450 6277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cbbbecf96b646009403a0c091c8a27b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:18.610657 6461 tablet_service.cc:1467] Processing CreateTablet for tablet 4cbbbecf96b646009403a0c091c8a27b (DEFAULT_TABLE table=test-table [id=f5611bcd2ed44cb49094c6231b89a89c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:41:18.610653 6070 tablet_service.cc:1467] Processing CreateTablet for tablet 4cbbbecf96b646009403a0c091c8a27b (DEFAULT_TABLE table=test-table [id=f5611bcd2ed44cb49094c6231b89a89c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:41:18.612744 6070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cbbbecf96b646009403a0c091c8a27b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:18.615470 6461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cbbbecf96b646009403a0c091c8a27b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:18.631273 6584 tablet_bootstrap.cc:492] T 4cbbbecf96b646009403a0c091c8a27b P c7e07a5c706a46e7b64c2365347d8dea: Bootstrap starting.
I20260810 18:41:18.636893 6585 tablet_bootstrap.cc:492] T 4cbbbecf96b646009403a0c091c8a27b P b468cba8f1d041a88f09a09beba8d713: Bootstrap starting.
I20260810 18:41:18.639135 6584 tablet_bootstrap.cc:654] T 4cbbbecf96b646009403a0c091c8a27b P c7e07a5c706a46e7b64c2365347d8dea: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:18.640759 6586 tablet_bootstrap.cc:492] T 4cbbbecf96b646009403a0c091c8a27b P 7279f4ac4a7d46439ff82fc1a11babfa: Bootstrap starting.
I20260810 18:41:18.642982 6584 log.cc:824] T 4cbbbecf96b646009403a0c091c8a27b P c7e07a5c706a46e7b64c2365347d8dea: Log is configured to *not* fsync() on all Append() calls
I20260810 18:41:18.644220 6585 tablet_bootstrap.cc:654] T 4cbbbecf96b646009403a0c091c8a27b P b468cba8f1d041a88f09a09beba8d713: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:18.646263 6586 tablet_bootstrap.cc:654] T 4cbbbecf96b646009403a0c091c8a27b P 7279f4ac4a7d46439ff82fc1a11babfa: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:18.646914 6585 log.cc:824] T 4cbbbecf96b646009403a0c091c8a27b P b468cba8f1d041a88f09a09beba8d713: Log is configured to *not* fsync() on all Append() calls
I20260810 18:41:18.648602 6586 log.cc:824] T 4cbbbecf96b646009403a0c091c8a27b P 7279f4ac4a7d46439ff82fc1a11babfa: Log is configured to *not* fsync() on all Append() calls
I20260810 18:41:18.648804 6584 tablet_bootstrap.cc:492] T 4cbbbecf96b646009403a0c091c8a27b P c7e07a5c706a46e7b64c2365347d8dea: No bootstrap required, opened a new log
I20260810 18:41:18.649274 6584 ts_tablet_manager.cc:1397] T 4cbbbecf96b646009403a0c091c8a27b P c7e07a5c706a46e7b64c2365347d8dea: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.009s
I20260810 18:41:18.657917 6586 tablet_bootstrap.cc:492] T 4cbbbecf96b646009403a0c091c8a27b P 7279f4ac4a7d46439ff82fc1a11babfa: No bootstrap required, opened a new log
I20260810 18:41:18.658412 6586 ts_tablet_manager.cc:1397] T 4cbbbecf96b646009403a0c091c8a27b P 7279f4ac4a7d46439ff82fc1a11babfa: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.011s
I20260810 18:41:18.660682 6585 tablet_bootstrap.cc:492] T 4cbbbecf96b646009403a0c091c8a27b P b468cba8f1d041a88f09a09beba8d713: No bootstrap required, opened a new log
I20260810 18:41:18.661135 6585 ts_tablet_manager.cc:1397] T 4cbbbecf96b646009403a0c091c8a27b P b468cba8f1d041a88f09a09beba8d713: Time spent bootstrapping tablet: real 0.025s user 0.000s sys 0.018s
I20260810 18:41:18.666268 6584 raft_consensus.cc:348] T 4cbbbecf96b646009403a0c091c8a27b P c7e07a5c706a46e7b64c2365347d8dea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7e07a5c706a46e7b64c2365347d8dea" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 40531 } } peers { permanent_uuid: "b468cba8f1d041a88f09a09beba8d713" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45777 } } peers { permanent_uuid: "7279f4ac4a7d46439ff82fc1a11babfa" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 33401 } }
I20260810 18:41:18.667066 6584 raft_consensus.cc:374] T 4cbbbecf96b646009403a0c091c8a27b P c7e07a5c706a46e7b64c2365347d8dea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:18.667246 6584 raft_consensus.cc:729] T 4cbbbecf96b646009403a0c091c8a27b P c7e07a5c706a46e7b64c2365347d8dea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7e07a5c706a46e7b64c2365347d8dea, State: Initialized, Role: FOLLOWER
I20260810 18:41:18.668162 6584 consensus_queue.cc:260] T 4cbbbecf96b646009403a0c091c8a27b P c7e07a5c706a46e7b64c2365347d8dea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7e07a5c706a46e7b64c2365347d8dea" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 40531 } } peers { permanent_uuid: "b468cba8f1d041a88f09a09beba8d713" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45777 } } peers { permanent_uuid: "7279f4ac4a7d46439ff82fc1a11babfa" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 33401 } }
I20260810 18:41:18.670295 6362 heartbeater.cc:503] Master 127.4.228.254:37085 was elected leader, sending a full tablet report...
I20260810 18:41:18.671258 6584 ts_tablet_manager.cc:1428] T 4cbbbecf96b646009403a0c091c8a27b P c7e07a5c706a46e7b64c2365347d8dea: Time spent starting tablet: real 0.022s user 0.014s sys 0.009s
I20260810 18:41:18.674547 6586 raft_consensus.cc:348] T 4cbbbecf96b646009403a0c091c8a27b P 7279f4ac4a7d46439ff82fc1a11babfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7e07a5c706a46e7b64c2365347d8dea" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 40531 } } peers { permanent_uuid: "b468cba8f1d041a88f09a09beba8d713" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45777 } } peers { permanent_uuid: "7279f4ac4a7d46439ff82fc1a11babfa" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 33401 } }
I20260810 18:41:18.674782 6585 raft_consensus.cc:348] T 4cbbbecf96b646009403a0c091c8a27b P b468cba8f1d041a88f09a09beba8d713 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7e07a5c706a46e7b64c2365347d8dea" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 40531 } } peers { permanent_uuid: "b468cba8f1d041a88f09a09beba8d713" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45777 } } peers { permanent_uuid: "7279f4ac4a7d46439ff82fc1a11babfa" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 33401 } }
I20260810 18:41:18.675312 6585 raft_consensus.cc:374] T 4cbbbecf96b646009403a0c091c8a27b P b468cba8f1d041a88f09a09beba8d713 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:18.675321 6586 raft_consensus.cc:374] T 4cbbbecf96b646009403a0c091c8a27b P 7279f4ac4a7d46439ff82fc1a11babfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:18.675489 6585 raft_consensus.cc:729] T 4cbbbecf96b646009403a0c091c8a27b P b468cba8f1d041a88f09a09beba8d713 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b468cba8f1d041a88f09a09beba8d713, State: Initialized, Role: FOLLOWER
I20260810 18:41:18.675554 6586 raft_consensus.cc:729] T 4cbbbecf96b646009403a0c091c8a27b P 7279f4ac4a7d46439ff82fc1a11babfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7279f4ac4a7d46439ff82fc1a11babfa, State: Initialized, Role: FOLLOWER
I20260810 18:41:18.676288 6585 consensus_queue.cc:260] T 4cbbbecf96b646009403a0c091c8a27b P b468cba8f1d041a88f09a09beba8d713 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7e07a5c706a46e7b64c2365347d8dea" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 40531 } } peers { permanent_uuid: "b468cba8f1d041a88f09a09beba8d713" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45777 } } peers { permanent_uuid: "7279f4ac4a7d46439ff82fc1a11babfa" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 33401 } }
I20260810 18:41:18.676646 6586 consensus_queue.cc:260] T 4cbbbecf96b646009403a0c091c8a27b P 7279f4ac4a7d46439ff82fc1a11babfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7e07a5c706a46e7b64c2365347d8dea" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 40531 } } peers { permanent_uuid: "b468cba8f1d041a88f09a09beba8d713" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45777 } } peers { permanent_uuid: "7279f4ac4a7d46439ff82fc1a11babfa" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 33401 } }
I20260810 18:41:18.679237 6585 ts_tablet_manager.cc:1428] T 4cbbbecf96b646009403a0c091c8a27b P b468cba8f1d041a88f09a09beba8d713: Time spent starting tablet: real 0.018s user 0.019s sys 0.001s
I20260810 18:41:18.679934 6557 heartbeater.cc:503] Master 127.4.228.254:37085 was elected leader, sending a full tablet report...
I20260810 18:41:18.681218 6586 ts_tablet_manager.cc:1428] T 4cbbbecf96b646009403a0c091c8a27b P 7279f4ac4a7d46439ff82fc1a11babfa: Time spent starting tablet: real 0.023s user 0.007s sys 0.014s
W20260810 18:41:18.707758 6558 tablet.cc:2369] T 4cbbbecf96b646009403a0c091c8a27b P 7279f4ac4a7d46439ff82fc1a11babfa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:41:18.815227 6172 tablet.cc:2369] T 4cbbbecf96b646009403a0c091c8a27b P b468cba8f1d041a88f09a09beba8d713: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:41:18.866664 6594 raft_consensus.cc:482] T 4cbbbecf96b646009403a0c091c8a27b P 7279f4ac4a7d46439ff82fc1a11babfa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:41:18.867192 6594 raft_consensus.cc:504] T 4cbbbecf96b646009403a0c091c8a27b P 7279f4ac4a7d46439ff82fc1a11babfa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7e07a5c706a46e7b64c2365347d8dea" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 40531 } } peers { permanent_uuid: "b468cba8f1d041a88f09a09beba8d713" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45777 } } peers { permanent_uuid: "7279f4ac4a7d46439ff82fc1a11babfa" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 33401 } }
I20260810 18:41:18.870201 6594 leader_election.cc:288] T 4cbbbecf96b646009403a0c091c8a27b P 7279f4ac4a7d46439ff82fc1a11babfa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7e07a5c706a46e7b64c2365347d8dea (127.4.228.194:40531), b468cba8f1d041a88f09a09beba8d713 (127.4.228.193:45777)
I20260810 18:41:18.880393 6575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:18.871091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:44739 (local address 127.4.228.194:40531)
0810 18:41:18.871412 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:18.871419 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:18.871443 (+ 24us) server_negotiation.cc:407] Connection header received
0810 18:41:18.871530 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:18.871535 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:18.871652 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:18.871791 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:41:18.872501 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:18.874411 (+ 1910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:18.877793 (+ 3382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:18.879118 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:18.879310 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:18.879401 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:18.879961 (+ 560us) server_negotiation.cc:299] Negotiation successful
0810 18:41:18.880084 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 18:41:18.880676 6560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:18.870449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:40531 (local address 127.4.228.195:44739)
0810 18:41:18.870767 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:18.870788 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:18.870947 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:18.871823 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:18.871829 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:18.871859 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:41:18.872333 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:18.872348 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:18.874625 (+ 2277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:18.874637 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:18.877563 (+ 2926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:18.877578 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:18.879307 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:18.879315 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:18.879711 (+ 396us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:18.879774 (+ 63us) client_negotiation.cc:769] Sending connection context
0810 18:41:18.879901 (+ 127us) client_negotiation.cc:240] Negotiation successful
0810 18:41:18.880115 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"mutex_wait_us":29}
I20260810 18:41:18.883011 6580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:18.871356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:50571 (local address 127.4.228.193:45777)
0810 18:41:18.871624 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:18.871631 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:18.872767 (+ 1136us) server_negotiation.cc:407] Connection header received
0810 18:41:18.873240 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:18.873249 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:18.873374 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:18.873496 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:41:18.875218 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:18.876900 (+ 1682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:18.879881 (+ 2981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:18.881597 (+ 1716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:18.881833 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:18.881940 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:18.882258 (+ 318us) server_negotiation.cc:299] Negotiation successful
0810 18:41:18.882716 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 18:41:18.882915 6306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cbbbecf96b646009403a0c091c8a27b" candidate_uuid: "7279f4ac4a7d46439ff82fc1a11babfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7e07a5c706a46e7b64c2365347d8dea" is_pre_election: true
I20260810 18:41:18.883491 6595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:18.871086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:45777 (local address 127.4.228.195:50571)
0810 18:41:18.872620 (+ 1534us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:18.872641 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:18.872840 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:18.873554 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:18.873562 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:18.873594 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:41:18.874973 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:18.874988 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:18.877128 (+ 2140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:18.877133 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:18.879662 (+ 2529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:18.879676 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:18.881907 (+ 2231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:18.881913 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:18.882073 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:18.882099 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 18:41:18.882223 (+ 124us) client_negotiation.cc:240] Negotiation successful
0810 18:41:18.882362 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1348,"mutex_wait_us":35,"thread_start_us":904,"threads_started":1}
I20260810 18:41:18.884311 6306 raft_consensus.cc:2454] T 4cbbbecf96b646009403a0c091c8a27b P c7e07a5c706a46e7b64c2365347d8dea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7279f4ac4a7d46439ff82fc1a11babfa in term 0.
I20260810 18:41:18.884835 6100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cbbbecf96b646009403a0c091c8a27b" candidate_uuid: "7279f4ac4a7d46439ff82fc1a11babfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b468cba8f1d041a88f09a09beba8d713" is_pre_election: true
I20260810 18:41:18.885700 6404 leader_election.cc:302] T 4cbbbecf96b646009403a0c091c8a27b P 7279f4ac4a7d46439ff82fc1a11babfa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7279f4ac4a7d46439ff82fc1a11babfa, c7e07a5c706a46e7b64c2365347d8dea; no voters:
I20260810 18:41:18.885679 6100 raft_consensus.cc:2454] T 4cbbbecf96b646009403a0c091c8a27b P b468cba8f1d041a88f09a09beba8d713 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7279f4ac4a7d46439ff82fc1a11babfa in term 0.
I20260810 18:41:18.886221 6594 raft_consensus.cc:2781] T 4cbbbecf96b646009403a0c091c8a27b P 7279f4ac4a7d46439ff82fc1a11babfa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:41:18.886406 6594 raft_consensus.cc:482] T 4cbbbecf96b646009403a0c091c8a27b P 7279f4ac4a7d46439ff82fc1a11babfa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:41:18.886603 6594 raft_consensus.cc:3037] T 4cbbbecf96b646009403a0c091c8a27b P 7279f4ac4a7d46439ff82fc1a11babfa [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:18.889689 6594 raft_consensus.cc:504] T 4cbbbecf96b646009403a0c091c8a27b P 7279f4ac4a7d46439ff82fc1a11babfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7e07a5c706a46e7b64c2365347d8dea" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 40531 } } peers { permanent_uuid: "b468cba8f1d041a88f09a09beba8d713" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45777 } } peers { permanent_uuid: "7279f4ac4a7d46439ff82fc1a11babfa" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 33401 } }
I20260810 18:41:18.890661 6594 leader_election.cc:288] T 4cbbbecf96b646009403a0c091c8a27b P 7279f4ac4a7d46439ff82fc1a11babfa [CANDIDATE]: Term 1 election: Requested vote from peers c7e07a5c706a46e7b64c2365347d8dea (127.4.228.194:40531), b468cba8f1d041a88f09a09beba8d713 (127.4.228.193:45777)
I20260810 18:41:18.891409 6306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cbbbecf96b646009403a0c091c8a27b" candidate_uuid: "7279f4ac4a7d46439ff82fc1a11babfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7e07a5c706a46e7b64c2365347d8dea"
I20260810 18:41:18.891417 6100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cbbbecf96b646009403a0c091c8a27b" candidate_uuid: "7279f4ac4a7d46439ff82fc1a11babfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b468cba8f1d041a88f09a09beba8d713"
I20260810 18:41:18.891754 6100 raft_consensus.cc:3037] T 4cbbbecf96b646009403a0c091c8a27b P b468cba8f1d041a88f09a09beba8d713 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:18.891682 6306 raft_consensus.cc:3037] T 4cbbbecf96b646009403a0c091c8a27b P c7e07a5c706a46e7b64c2365347d8dea [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:18.895639 6306 raft_consensus.cc:2454] T 4cbbbecf96b646009403a0c091c8a27b P c7e07a5c706a46e7b64c2365347d8dea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7279f4ac4a7d46439ff82fc1a11babfa in term 1.
I20260810 18:41:18.896322 6404 leader_election.cc:302] T 4cbbbecf96b646009403a0c091c8a27b P 7279f4ac4a7d46439ff82fc1a11babfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7279f4ac4a7d46439ff82fc1a11babfa, c7e07a5c706a46e7b64c2365347d8dea; no voters:
I20260810 18:41:18.896345 6100 raft_consensus.cc:2454] T 4cbbbecf96b646009403a0c091c8a27b P b468cba8f1d041a88f09a09beba8d713 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7279f4ac4a7d46439ff82fc1a11babfa in term 1.
I20260810 18:41:18.896788 6594 raft_consensus.cc:2781] T 4cbbbecf96b646009403a0c091c8a27b P 7279f4ac4a7d46439ff82fc1a11babfa [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:41:18.897675 6594 raft_consensus.cc:686] T 4cbbbecf96b646009403a0c091c8a27b P 7279f4ac4a7d46439ff82fc1a11babfa [term 1 LEADER]: Becoming Leader. State: Replica: 7279f4ac4a7d46439ff82fc1a11babfa, State: Running, Role: LEADER
W20260810 18:41:18.898599 6366 tablet.cc:2369] T 4cbbbecf96b646009403a0c091c8a27b P c7e07a5c706a46e7b64c2365347d8dea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:41:18.898511 6594 consensus_queue.cc:237] T 4cbbbecf96b646009403a0c091c8a27b P 7279f4ac4a7d46439ff82fc1a11babfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7e07a5c706a46e7b64c2365347d8dea" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 40531 } } peers { permanent_uuid: "b468cba8f1d041a88f09a09beba8d713" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45777 } } peers { permanent_uuid: "7279f4ac4a7d46439ff82fc1a11babfa" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 33401 } }
I20260810 18:41:18.905943 5889 catalog_manager.cc:5495] T 4cbbbecf96b646009403a0c091c8a27b P 7279f4ac4a7d46439ff82fc1a11babfa reported cstate change: term changed from 0 to 1, leader changed from <none> to 7279f4ac4a7d46439ff82fc1a11babfa (127.4.228.195). New cstate: current_term: 1 leader_uuid: "7279f4ac4a7d46439ff82fc1a11babfa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7e07a5c706a46e7b64c2365347d8dea" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 40531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b468cba8f1d041a88f09a09beba8d713" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7279f4ac4a7d46439ff82fc1a11babfa" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 33401 } health_report { overall_health: HEALTHY } } }
Aug 10 18:41:18 dist-test-slave-7kzw krb5kdc[5829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.193@KRBTEST.COM
I20260810 18:41:18.983431 6580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:18.959851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51212 (local address 127.4.228.193:45777)
0810 18:41:18.960093 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:18.960098 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:18.961191 (+ 1093us) server_negotiation.cc:407] Connection header received
0810 18:41:18.961424 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:18.961428 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:18.961556 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:18.961719 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:18.964497 (+ 2778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:18.965582 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:18.967241 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:18.967859 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:18.967965 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:18.974013 (+ 6048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:18.974036 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:18.974058 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:18.974105 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:18.979292 (+ 5187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:18.980369 (+ 1077us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:18.980379 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:18.980382 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:18.980676 (+ 294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:18.981307 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:18.981311 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:18.981313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:18.981891 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:18.982219 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:18.982688 (+ 469us) server_negotiation.cc:299] Negotiation successful
0810 18:41:18.982912 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
Aug 10 18:41:19 dist-test-slave-7kzw krb5kdc[5829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387279, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 18:41:19 dist-test-slave-7kzw krb5kdc[5829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387279, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.228.193@KRBTEST.COM
I20260810 18:41:19.053043 6580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:19.033191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51228 (local address 127.4.228.193:45777)
0810 18:41:19.033449 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:19.033459 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:19.033664 (+ 205us) server_negotiation.cc:407] Connection header received
0810 18:41:19.033851 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:19.033856 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:19.033935 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:19.034041 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:19.036249 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:19.037111 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:19.039003 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:19.039501 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:19.039617 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:19.045475 (+ 5858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:19.045481 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:19.045485 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:19.045531 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:19.049322 (+ 3791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:19.050325 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:19.050331 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:19.050334 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:19.050715 (+ 381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:19.051202 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:19.051207 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:19.051209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:19.051675 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:19.052040 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:19.052234 (+ 194us) server_negotiation.cc:299] Negotiation successful
0810 18:41:19.052482 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
Aug 10 18:41:19 dist-test-slave-7kzw krb5kdc[5829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 18:41:19 dist-test-slave-7kzw krb5kdc[5829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.193@KRBTEST.COM
I20260810 18:41:19.123281 6580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:19.101630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51234 (local address 127.4.228.193:45777)
0810 18:41:19.101897 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:19.101904 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:19.102217 (+ 313us) server_negotiation.cc:407] Connection header received
0810 18:41:19.102512 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:19.102517 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:19.102594 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:19.102696 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:19.104323 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:19.105452 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:19.107217 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:19.107715 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:19.107833 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:19.114902 (+ 7069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:19.114909 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:19.114913 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:19.114966 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:19.119018 (+ 4052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:19.120064 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:19.120069 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:19.120072 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:19.120286 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:19.120992 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:19.120996 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:19.120998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:19.121542 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:19.121887 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:19.122231 (+ 344us) server_negotiation.cc:299] Negotiation successful
0810 18:41:19.122713 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
W20260810 18:41:19.126041 6040 tablet_service.cc:2908] Rejecting scan request for tablet 4cbbbecf96b646009403a0c091c8a27b: Uninitialized: clean time has not yet been initialized
I20260810 18:41:19.130277 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 5983
I20260810 18:41:19.149137 6580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:19.131597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51240 (local address 127.4.228.193:45777)
0810 18:41:19.131860 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:19.131866 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:19.132031 (+ 165us) server_negotiation.cc:407] Connection header received
0810 18:41:19.132184 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:19.132188 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:19.132285 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:19.132376 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:19.134052 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:19.134892 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:19.136725 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:19.137197 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:19.137312 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:19.140855 (+ 3543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:19.140860 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:19.140864 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:19.140906 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:19.144764 (+ 3858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:19.146018 (+ 1254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:19.146023 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:19.146026 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:19.146243 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:19.146862 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:19.146868 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:19.146872 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:19.147228 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:19.147909 (+ 681us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:19.148254 (+ 345us) server_negotiation.cc:299] Negotiation successful
0810 18:41:19.148603 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260810 18:41:19.150024 6162 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:41:19.249428 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 5983
I20260810 18:41:19.283219 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 6181
Aug 10 18:41:19 dist-test-slave-7kzw krb5kdc[5829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.194@KRBTEST.COM
I20260810 18:41:19.301016 6575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:19.284266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54462 (local address 127.4.228.194:40531)
0810 18:41:19.284457 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:19.284462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:19.284482 (+ 20us) server_negotiation.cc:407] Connection header received
0810 18:41:19.284555 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:19.284559 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:19.284631 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:19.284746 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:19.286078 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:19.286978 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:19.288383 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:19.288819 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:19.288931 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:19.294212 (+ 5281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:19.294232 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:19.294259 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:19.294309 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:19.297777 (+ 3468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:19.298655 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:19.298698 (+ 43us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:19.298701 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:19.298852 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:19.299363 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:19.299368 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:19.299370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:19.299906 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:19.300182 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:19.300551 (+ 369us) server_negotiation.cc:299] Negotiation successful
0810 18:41:19.300742 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 18:41:19.301783 6356 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:41:19.359936 6595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:19.359346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:45777 (local address 127.4.228.195:43981)
0810 18:41:19.359553 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:19.359745 (+ 192us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.193:45777: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260810 18:41:19.360307 6404 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.228.193:45777: connect: Connection refused (error 111)
I20260810 18:41:19.362105 6595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:19.361582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:45777 (local address 127.4.228.195:56265)
0810 18:41:19.361752 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:19.361868 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.193:45777: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 18:41:19.362517 6404 consensus_peers.cc:487] T 4cbbbecf96b646009403a0c091c8a27b P 7279f4ac4a7d46439ff82fc1a11babfa -> Peer b468cba8f1d041a88f09a09beba8d713 (127.4.228.193:45777): Couldn't send request to peer b468cba8f1d041a88f09a09beba8d713. Status: Network error: Client connection negotiation failed: client connection to 127.4.228.193:45777: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 18:41:19.393532 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 6181
I20260810 18:41:19.423367 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 6380
I20260810 18:41:19.432324 6595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:19.431815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:40531 (local address 127.4.228.195:56679)
0810 18:41:19.432026 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:19.432160 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.194:40531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
Aug 10 18:41:19 dist-test-slave-7kzw krb5kdc[5829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.195@KRBTEST.COM
I20260810 18:41:19.433552 6595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:19.433129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:40531 (local address 127.4.228.195:60255)
0810 18:41:19.433325 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:19.433406 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.194:40531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260810 18:41:19.433764 6404 consensus_peers.cc:487] T 4cbbbecf96b646009403a0c091c8a27b P 7279f4ac4a7d46439ff82fc1a11babfa -> Peer c7e07a5c706a46e7b64c2365347d8dea (127.4.228.194:40531): Couldn't send request to peer c7e07a5c706a46e7b64c2365347d8dea. Status: Network error: Client connection negotiation failed: client connection to 127.4.228.194:40531: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 18:41:19.440018 6625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:19.424304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55550 (local address 127.4.228.195:33401)
0810 18:41:19.424848 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:19.424853 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:19.424875 (+ 22us) server_negotiation.cc:407] Connection header received
0810 18:41:19.424931 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:19.424934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:19.425000 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:19.425122 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:19.426448 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:19.427199 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:19.428548 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:19.428959 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:19.429095 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:19.433644 (+ 4549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:19.433672 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:19.433696 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:19.433740 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:19.436798 (+ 3058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:19.437544 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:19.437554 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:19.437556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:19.437715 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:19.438113 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:19.438118 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:19.438121 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:19.438729 (+ 608us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:19.439060 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:19.439387 (+ 327us) server_negotiation.cc:299] Negotiation successful
0810 18:41:19.439639 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":257,"threads_started":1}
I20260810 18:41:19.440771 6551 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:41:19.534040 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 6380
I20260810 18:41:19.561010 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 5846
I20260810 18:41:19.562541 5942 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:41:19.679772 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 5846
2026-08-10T18:41:19Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (5143 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:41:19 dist-test-slave-7kzw krb5kdc[6632](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:41:19 dist-test-slave-7kzw krb5kdc[6632](info): set up 2 sockets
Aug 10 18:41:19 dist-test-slave-7kzw krb5kdc[6632](info): commencing operation
krb5kdc: starting...
W20260810 18:41:21.067858 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.332s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:41:21 dist-test-slave-7kzw krb5kdc[6632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:41:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:41:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.228.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
I20260810 18:41:21.330704 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:45135
--webserver_interface=127.4.228.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:40555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:45135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:41:21.699369 6647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:41:21.699697 6647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:41:21.699790 6647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:41:21.707218 6647 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:41:21.707368 6647 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 18:41:21.707412 6647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:41:21.707469 6647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:41:21.707521 6647 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:41:21.707588 6647 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:41:21.718079 6647 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:40555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:45135
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:45135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:41:21.719586 6647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:41:21.721017 6647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:41:21.734579 6652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:21.734483 6659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:21.735095 6655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:21.735675 6647 server_base.cc:1034] running on GCE node
I20260810 18:41:21.736685 6647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:41:21.738409 6647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:41:21.739734 6647 hybrid_clock.cc:648] HybridClock initialized: now 1786387281739635 us; error 63 us; skew 500 ppm
I20260810 18:41:21.740139 6647 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:41:21 dist-test-slave-7kzw krb5kdc[6632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387281, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:41:21.743758 6647 init.cc:377] Logged in from keytab as kudu/127.4.228.254@KRBTEST.COM (short username kudu)
I20260810 18:41:21.745597 6647 webserver.cc:460] Webserver started at http://127.4.228.254:40299/ using document root <none> and password file <none>
I20260810 18:41:21.746372 6647 fs_manager.cc:359] Metadata directory not provided
I20260810 18:41:21.746567 6647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:41:21.746919 6647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:41:21.750381 6647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "5a659e0d5d194cfa93aa8c8d12e9ccf3"
format_stamp: "Formatted at 2026-08-10 18:41:21 on dist-test-slave-7kzw"
server_key: "4c5d2b39a15438ae1723e18f43abbc39"
server_key_iv: "fafcd684ad51c1b80268c557fc5d2cc8"
server_key_version: "encryptionkey@0"
I20260810 18:41:21.751343 6647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "5a659e0d5d194cfa93aa8c8d12e9ccf3"
format_stamp: "Formatted at 2026-08-10 18:41:21 on dist-test-slave-7kzw"
server_key: "4c5d2b39a15438ae1723e18f43abbc39"
server_key_iv: "fafcd684ad51c1b80268c557fc5d2cc8"
server_key_version: "encryptionkey@0"
I20260810 18:41:21.757481 6647 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260810 18:41:21.761663 6668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:21.763391 6647 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:41:21.763557 6647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "5a659e0d5d194cfa93aa8c8d12e9ccf3"
format_stamp: "Formatted at 2026-08-10 18:41:21 on dist-test-slave-7kzw"
server_key: "4c5d2b39a15438ae1723e18f43abbc39"
server_key_iv: "fafcd684ad51c1b80268c557fc5d2cc8"
server_key_version: "encryptionkey@0"
I20260810 18:41:21.763772 6647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:41:21.792181 6647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:41:21.797988 6647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:41:21.798259 6647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:41:21.823062 6647 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:45135
I20260810 18:41:21.823113 6763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:45135 every 8 connection(s)
I20260810 18:41:21.824584 6647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:41:21.827019 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 6647
I20260810 18:41:21.827315 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:41:21.828274 5011 external_mini_cluster.cc:1428] Setting key 667701138b7e12843d09cba569819613
I20260810 18:41:21.830070 6764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 18:41:21 dist-test-slave-7kzw krb5kdc[6632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:21.846716 6764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a659e0d5d194cfa93aa8c8d12e9ccf3: Bootstrap starting.
I20260810 18:41:21.851862 6764 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a659e0d5d194cfa93aa8c8d12e9ccf3: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:21.853298 6764 log.cc:824] T 00000000000000000000000000000000 P 5a659e0d5d194cfa93aa8c8d12e9ccf3: Log is configured to *not* fsync() on all Append() calls
I20260810 18:41:21.857553 6764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a659e0d5d194cfa93aa8c8d12e9ccf3: No bootstrap required, opened a new log
I20260810 18:41:21.857745 6771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:21.830828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40442 (local address 127.4.228.254:45135)
0810 18:41:21.831787 (+ 959us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:21.831800 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:21.831846 (+ 46us) server_negotiation.cc:407] Connection header received
0810 18:41:21.833642 (+ 1796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:21.833669 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:21.834223 (+ 554us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:21.834732 (+ 509us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:21.836911 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:21.838555 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:21.840579 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:21.841413 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:21.841585 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:21.847150 (+ 5565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:21.847164 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:21.847188 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:21.847253 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:21.851976 (+ 4723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:21.852807 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:21.852816 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:21.852826 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:21.853093 (+ 267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:21.853716 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:21.853722 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:21.853725 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:21.854743 (+ 1018us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:21.855010 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:21.855678 (+ 668us) server_negotiation.cc:299] Negotiation successful
0810 18:41:21.856187 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":721,"thread_start_us":472,"threads_started":1}
I20260810 18:41:21.867019 6764 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a659e0d5d194cfa93aa8c8d12e9ccf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a659e0d5d194cfa93aa8c8d12e9ccf3" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45135 } }
I20260810 18:41:21.867508 6764 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a659e0d5d194cfa93aa8c8d12e9ccf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:21.867636 6764 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a659e0d5d194cfa93aa8c8d12e9ccf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a659e0d5d194cfa93aa8c8d12e9ccf3, State: Initialized, Role: FOLLOWER
I20260810 18:41:21.868458 6764 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a659e0d5d194cfa93aa8c8d12e9ccf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a659e0d5d194cfa93aa8c8d12e9ccf3" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45135 } }
I20260810 18:41:21.868801 6764 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a659e0d5d194cfa93aa8c8d12e9ccf3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:41:21.868976 6764 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a659e0d5d194cfa93aa8c8d12e9ccf3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:41:21.869239 6764 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a659e0d5d194cfa93aa8c8d12e9ccf3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:21.871799 6764 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a659e0d5d194cfa93aa8c8d12e9ccf3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a659e0d5d194cfa93aa8c8d12e9ccf3" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45135 } }
I20260810 18:41:21.872375 6764 leader_election.cc:302] T 00000000000000000000000000000000 P 5a659e0d5d194cfa93aa8c8d12e9ccf3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a659e0d5d194cfa93aa8c8d12e9ccf3; no voters:
I20260810 18:41:21.873092 6764 leader_election.cc:288] T 00000000000000000000000000000000 P 5a659e0d5d194cfa93aa8c8d12e9ccf3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:41:21.873226 6774 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a659e0d5d194cfa93aa8c8d12e9ccf3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:41:21.874233 6774 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a659e0d5d194cfa93aa8c8d12e9ccf3 [term 1 LEADER]: Becoming Leader. State: Replica: 5a659e0d5d194cfa93aa8c8d12e9ccf3, State: Running, Role: LEADER
I20260810 18:41:21.874840 6774 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a659e0d5d194cfa93aa8c8d12e9ccf3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a659e0d5d194cfa93aa8c8d12e9ccf3" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45135 } }
I20260810 18:41:21.875768 6764 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a659e0d5d194cfa93aa8c8d12e9ccf3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:41:21.878001 6777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a659e0d5d194cfa93aa8c8d12e9ccf3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a659e0d5d194cfa93aa8c8d12e9ccf3. Latest consensus state: current_term: 1 leader_uuid: "5a659e0d5d194cfa93aa8c8d12e9ccf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a659e0d5d194cfa93aa8c8d12e9ccf3" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45135 } } }
I20260810 18:41:21.878407 6777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a659e0d5d194cfa93aa8c8d12e9ccf3 [sys.catalog]: This master's current role is: LEADER
I20260810 18:41:21.879283 6775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a659e0d5d194cfa93aa8c8d12e9ccf3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a659e0d5d194cfa93aa8c8d12e9ccf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a659e0d5d194cfa93aa8c8d12e9ccf3" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45135 } } }
I20260810 18:41:21.879590 6775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a659e0d5d194cfa93aa8c8d12e9ccf3 [sys.catalog]: This master's current role is: LEADER
I20260810 18:41:21.880976 6781 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:41:21.886690 6781 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:41:21.896713 6781 catalog_manager.cc:1347] Generated new cluster ID: a214a123978e44c4b1a4390d2a1f5166
I20260810 18:41:21.896845 6781 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:41:21.920166 6781 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:41:21.921741 6781 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:41:21.930964 6781 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a659e0d5d194cfa93aa8c8d12e9ccf3: Generated new TSK 0
I20260810 18:41:21.931951 6781 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.228.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab.
Entry for principal kudu/127.4.228.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab.
I20260810 18:41:21.995721 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.193:0
--local_ip_for_outbound_sockets=127.4.228.193
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:45135
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:40555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:41:22.365716 6805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:41:22.366036 6805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:41:22.366199 6805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:41:22.373813 6805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:41:22.373941 6805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:41:22.374125 6805 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 18:41:22.374264 6805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.193
I20260810 18:41:22.385797 6805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:40555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.228.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.228.254:45135
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.193
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:41:22.387319 6805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:41:22.388813 6805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:41:22.404151 6812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:22.404342 6813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:22.404392 6816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:22.405081 6805 server_base.cc:1034] running on GCE node
I20260810 18:41:22.405857 6805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:41:22.407022 6805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:41:22.408267 6805 hybrid_clock.cc:648] HybridClock initialized: now 1786387282408187 us; error 48 us; skew 500 ppm
I20260810 18:41:22.408630 6805 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:41:22 dist-test-slave-7kzw krb5kdc[6632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387282, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:41:22.412501 6805 init.cc:377] Logged in from keytab as kudu/127.4.228.193@KRBTEST.COM (short username kudu)
I20260810 18:41:22.414340 6805 webserver.cc:460] Webserver started at http://127.4.228.193:46483/ using document root <none> and password file <none>
I20260810 18:41:22.415154 6805 fs_manager.cc:359] Metadata directory not provided
I20260810 18:41:22.415256 6805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:41:22.415632 6805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:41:22.418861 6805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-0/data/instance:
uuid: "444dd6af1e194cb9bb17cc9a351457ea"
format_stamp: "Formatted at 2026-08-10 18:41:22 on dist-test-slave-7kzw"
server_key: "24e589439ae19a6e0dcf7fa28a51ac85"
server_key_iv: "dfe16ebabd2f237c02be60932adf0c9b"
server_key_version: "encryptionkey@0"
I20260810 18:41:22.419760 6805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance:
uuid: "444dd6af1e194cb9bb17cc9a351457ea"
format_stamp: "Formatted at 2026-08-10 18:41:22 on dist-test-slave-7kzw"
server_key: "24e589439ae19a6e0dcf7fa28a51ac85"
server_key_iv: "dfe16ebabd2f237c02be60932adf0c9b"
server_key_version: "encryptionkey@0"
I20260810 18:41:22.425384 6805 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 18:41:22.429450 6829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:22.431280 6805 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260810 18:41:22.431478 6805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-0/wal
uuid: "444dd6af1e194cb9bb17cc9a351457ea"
format_stamp: "Formatted at 2026-08-10 18:41:22 on dist-test-slave-7kzw"
server_key: "24e589439ae19a6e0dcf7fa28a51ac85"
server_key_iv: "dfe16ebabd2f237c02be60932adf0c9b"
server_key_version: "encryptionkey@0"
I20260810 18:41:22.431722 6805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:41:22.445809 6805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:41:22.450248 6805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:41:22.450599 6805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:41:22.453708 6805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:41:22.453825 6805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:22.453974 6805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:41:22.454049 6805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 18:41:22 dist-test-slave-7kzw krb5kdc[6632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387282, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:22.479319 6840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:22.454400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:42483 (local address 127.4.228.254:45135)
0810 18:41:22.457447 (+ 3047us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:22.457451 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:22.457470 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:41:22.460317 (+ 2847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:22.460325 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:22.460434 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:22.460580 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:22.463031 (+ 2451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:22.464117 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:22.466826 (+ 2709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:22.467719 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:22.467904 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:22.472323 (+ 4419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:22.472333 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:22.472339 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:22.472392 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:22.475918 (+ 3526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:22.476775 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:22.476781 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:22.476784 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:22.476915 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:22.477418 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:22.477428 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:22.477431 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:22.477716 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:22.477877 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:22.478710 (+ 833us) server_negotiation.cc:299] Negotiation successful
0810 18:41:22.478947 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2931,"thread_start_us":2821,"threads_started":1}
I20260810 18:41:22.480468 6841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:22.455295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45135 (local address 127.4.228.193:42483)
0810 18:41:22.456413 (+ 1118us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:22.456481 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:22.459847 (+ 3366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:22.460890 (+ 1043us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:22.460899 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:22.461747 (+ 848us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:22.462830 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:22.462852 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:22.464333 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:22.464342 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:22.466599 (+ 2257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:22.466616 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:22.468008 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:22.468021 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:22.468318 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:22.469227 (+ 909us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:41:22.469251 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:22.472024 (+ 2773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:22.476067 (+ 4043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:22.476077 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:22.476097 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:22.476642 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:22.477030 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:22.477035 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:22.477039 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:22.477294 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:22.477859 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:22.477869 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:22.478261 (+ 392us) client_negotiation.cc:769] Sending connection context
0810 18:41:22.478582 (+ 321us) client_negotiation.cc:240] Negotiation successful
0810 18:41:22.479190 (+ 608us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":756,"thread_start_us":468,"threads_started":1}
I20260810 18:41:22.511005 6840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:22.495308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:39339 (local address 127.4.228.254:45135)
0810 18:41:22.495630 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:22.495639 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:22.496681 (+ 1042us) server_negotiation.cc:407] Connection header received
0810 18:41:22.496847 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:22.496856 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:22.496977 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:22.497126 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:22.498263 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:22.499611 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:22.501124 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:22.502084 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:22.502264 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:22.505046 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:22.505051 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:22.505055 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:22.505091 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:22.508182 (+ 3091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:22.508907 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:22.508913 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:22.508922 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:22.509051 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:22.509432 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:22.509438 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:22.509442 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:22.509758 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:22.509919 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:22.510353 (+ 434us) server_negotiation.cc:299] Negotiation successful
0810 18:41:22.510628 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 18:41:22.511169 6958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:22.495314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45135 (local address 127.4.228.193:39339)
0810 18:41:22.496545 (+ 1231us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:22.496573 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:22.496720 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:22.497195 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:22.497200 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:22.497740 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:22.498089 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:22.498100 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:22.499802 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:22.499809 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:22.500959 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:22.500973 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:22.502315 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:22.502321 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:22.502459 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:22.503335 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:41:22.503386 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:22.504824 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:22.508333 (+ 3509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:22.508339 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:22.508344 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:22.508781 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:22.509173 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:22.509179 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:22.509183 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:22.509319 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:22.509881 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:22.509886 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:22.510084 (+ 198us) client_negotiation.cc:769] Sending connection context
0810 18:41:22.510235 (+ 151us) client_negotiation.cc:240] Negotiation successful
0810 18:41:22.510673 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1058,"thread_start_us":828,"threads_started":1}
I20260810 18:41:22.518148 6805 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.193:45063
I20260810 18:41:22.518244 7010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.193:45063 every 8 connection(s)
I20260810 18:41:22.519531 6805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
I20260810 18:41:22.521999 7011 heartbeater.cc:347] Connected to a master server at 127.4.228.254:45135
I20260810 18:41:22.522286 7011 heartbeater.cc:464] Registering TS with master...
I20260810 18:41:22.522513 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 6805
I20260810 18:41:22.522846 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance
I20260810 18:41:22.523365 7011 heartbeater.cc:511] Master 127.4.228.254:45135 requested a full tablet report, sending...
I20260810 18:41:22.523669 5011 external_mini_cluster.cc:1428] Setting key 0ecfa369b0cbb04427e55588a07b86af
I20260810 18:41:22.526000 6694 ts_manager.cc:194] Registered new tserver with Master: 444dd6af1e194cb9bb17cc9a351457ea (127.4.228.193:45063)
I20260810 18:41:22.528656 6694 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.193@KRBTEST.COM'} at 127.4.228.193:42483
WARNING: no policy specified for kudu/127.4.228.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab.
Entry for principal kudu/127.4.228.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab.
I20260810 18:41:22.582682 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.194:0
--local_ip_for_outbound_sockets=127.4.228.194
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:45135
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:40555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:41:22.980233 7016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:41:22.980603 7016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:41:22.980779 7016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:41:22.988456 7016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:41:22.988636 7016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:41:22.988790 7016 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 18:41:22.988929 7016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.194
I20260810 18:41:22.999204 7016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:40555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.228.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.228.254:45135
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.194
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:41:23.000630 7016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:41:23.002101 7016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:41:23.016640 7026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:23.016716 7028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:23.016793 7025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:23.017774 7016 server_base.cc:1034] running on GCE node
I20260810 18:41:23.018635 7016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:41:23.019753 7016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:41:23.021019 7016 hybrid_clock.cc:648] HybridClock initialized: now 1786387283020941 us; error 39 us; skew 500 ppm
I20260810 18:41:23.021430 7016 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:41:23 dist-test-slave-7kzw krb5kdc[6632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387283, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:41:23.028767 7016 init.cc:377] Logged in from keytab as kudu/127.4.228.194@KRBTEST.COM (short username kudu)
I20260810 18:41:23.030933 7016 webserver.cc:460] Webserver started at http://127.4.228.194:43505/ using document root <none> and password file <none>
I20260810 18:41:23.031852 7016 fs_manager.cc:359] Metadata directory not provided
I20260810 18:41:23.032007 7016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:41:23.032428 7016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:41:23.036059 7016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-1/data/instance:
uuid: "54b83db5272749afbdc9a6d659eed5d2"
format_stamp: "Formatted at 2026-08-10 18:41:23 on dist-test-slave-7kzw"
server_key: "a1650770e0ec9d15f1be23cbc923b572"
server_key_iv: "1f7e40554e007556aca409a1fbd4a5d0"
server_key_version: "encryptionkey@0"
I20260810 18:41:23.037065 7016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance:
uuid: "54b83db5272749afbdc9a6d659eed5d2"
format_stamp: "Formatted at 2026-08-10 18:41:23 on dist-test-slave-7kzw"
server_key: "a1650770e0ec9d15f1be23cbc923b572"
server_key_iv: "1f7e40554e007556aca409a1fbd4a5d0"
server_key_version: "encryptionkey@0"
I20260810 18:41:23.043164 7016 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260810 18:41:23.047315 7039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:23.048902 7016 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.003s
I20260810 18:41:23.049157 7016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-1/wal
uuid: "54b83db5272749afbdc9a6d659eed5d2"
format_stamp: "Formatted at 2026-08-10 18:41:23 on dist-test-slave-7kzw"
server_key: "a1650770e0ec9d15f1be23cbc923b572"
server_key_iv: "1f7e40554e007556aca409a1fbd4a5d0"
server_key_version: "encryptionkey@0"
I20260810 18:41:23.049432 7016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:41:23.075543 7016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:41:23.080705 7016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:41:23.081033 7016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:41:23.084589 7016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:41:23.084724 7016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:23.084903 7016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:41:23.084990 7016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 18:41:23 dist-test-slave-7kzw krb5kdc[6632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387283, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:23.116752 7062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:23.085485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:52643 (local address 127.4.228.254:45135)
0810 18:41:23.086112 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:23.086123 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:23.087664 (+ 1541us) server_negotiation.cc:407] Connection header received
0810 18:41:23.091138 (+ 3474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:23.091144 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:23.091235 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:23.091361 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:23.094395 (+ 3034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:23.095638 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:23.099833 (+ 4195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:23.100377 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:23.100512 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:23.106821 (+ 6309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:23.106832 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:23.106842 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:23.106909 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:23.111990 (+ 5081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:23.113350 (+ 1360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:23.113355 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:23.113357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:23.113562 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:23.114208 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:23.114214 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:23.114217 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:23.114662 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:23.114870 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:23.116162 (+ 1292us) server_negotiation.cc:299] Negotiation successful
0810 18:41:23.116390 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":314,"threads_started":1}
I20260810 18:41:23.119195 7065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:23.086190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45135 (local address 127.4.228.194:52643)
0810 18:41:23.087407 (+ 1217us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:23.087491 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:23.090662 (+ 3171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:23.091763 (+ 1101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:23.091787 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:23.092822 (+ 1035us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:23.094098 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:23.094146 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:23.095905 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:23.095914 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:23.099536 (+ 3622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:23.099563 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:23.100707 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:23.100720 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:23.101160 (+ 440us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:23.102316 (+ 1156us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:41:23.102372 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:23.106438 (+ 4066us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:23.112211 (+ 5773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:23.112228 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:23.112258 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:23.113138 (+ 880us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:23.113754 (+ 616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:23.113762 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:23.113771 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:23.114047 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:23.114850 (+ 803us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:23.114864 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:23.115560 (+ 696us) client_negotiation.cc:769] Sending connection context
0810 18:41:23.116024 (+ 464us) client_negotiation.cc:240] Negotiation successful
0810 18:41:23.116888 (+ 864us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":786,"thread_start_us":510,"threads_started":1}
I20260810 18:41:23.146031 7062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:23.126751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:37963 (local address 127.4.228.254:45135)
0810 18:41:23.127000 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:23.127010 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:23.128281 (+ 1271us) server_negotiation.cc:407] Connection header received
0810 18:41:23.128653 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:23.128660 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:23.128752 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:23.128876 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:23.130053 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:23.131312 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:23.134285 (+ 2973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:23.134836 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:23.135013 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:23.138000 (+ 2987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:23.138007 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:23.138012 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:23.138065 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:23.142330 (+ 4265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:23.143390 (+ 1060us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:23.143397 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:23.143400 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:23.143572 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:23.144181 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:23.144186 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:23.144189 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:23.144668 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:23.144845 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:23.145337 (+ 492us) server_negotiation.cc:299] Negotiation successful
0810 18:41:23.145628 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 18:41:23.146436 7163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:23.127108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45135 (local address 127.4.228.194:37963)
0810 18:41:23.128129 (+ 1021us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:23.128158 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:23.128385 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:23.128917 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:23.128922 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:23.129498 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:23.129865 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:23.129877 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:23.131529 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:23.131539 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:23.134061 (+ 2522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:23.134086 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:23.135374 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:23.135384 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:23.135521 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:23.136446 (+ 925us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:41:23.136482 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:23.137752 (+ 1270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:23.142534 (+ 4782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:23.142546 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:23.142553 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:23.143141 (+ 588us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:23.143727 (+ 586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:23.143731 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:23.143734 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:23.144002 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:23.144833 (+ 831us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:23.144838 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:23.145059 (+ 221us) client_negotiation.cc:769] Sending connection context
0810 18:41:23.145239 (+ 180us) client_negotiation.cc:240] Negotiation successful
0810 18:41:23.145678 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":815,"thread_start_us":589,"threads_started":1}
I20260810 18:41:23.147732 7225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.194:33349 every 8 connection(s)
I20260810 18:41:23.147732 7016 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.194:33349
I20260810 18:41:23.149801 7016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
I20260810 18:41:23.152467 7227 heartbeater.cc:347] Connected to a master server at 127.4.228.254:45135
I20260810 18:41:23.152755 7227 heartbeater.cc:464] Registering TS with master...
I20260810 18:41:23.153472 7227 heartbeater.cc:511] Master 127.4.228.254:45135 requested a full tablet report, sending...
I20260810 18:41:23.155246 6694 ts_manager.cc:194] Registered new tserver with Master: 54b83db5272749afbdc9a6d659eed5d2 (127.4.228.194:33349)
I20260810 18:41:23.156100 6694 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.194@KRBTEST.COM'} at 127.4.228.194:52643
I20260810 18:41:23.159864 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 7016
I20260810 18:41:23.160073 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance
I20260810 18:41:23.160543 5011 external_mini_cluster.cc:1428] Setting key 8b4f2d5acac6b73fdb9409e1e3099f58
WARNING: no policy specified for kudu/127.4.228.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab.
Entry for principal kudu/127.4.228.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab.
I20260810 18:41:23.215272 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.195:0
--local_ip_for_outbound_sockets=127.4.228.195
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:45135
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:40555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:41:23.532328 7011 heartbeater.cc:503] Master 127.4.228.254:45135 was elected leader, sending a full tablet report...
W20260810 18:41:23.582540 7235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:41:23.582892 7235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:41:23.582994 7235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:41:23.590417 7235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:41:23.590560 7235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:41:23.590679 7235 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 18:41:23.590765 7235 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.195
I20260810 18:41:23.600733 7235 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:40555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.228.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.228.254:45135
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.195
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:41:23.602128 7235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:41:23.603523 7235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:41:23.617231 7248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:23.617322 7255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:23.617376 7249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:23.618570 7235 server_base.cc:1034] running on GCE node
I20260810 18:41:23.619297 7235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:41:23.620353 7235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:41:23.621608 7235 hybrid_clock.cc:648] HybridClock initialized: now 1786387283621509 us; error 62 us; skew 500 ppm
I20260810 18:41:23.621963 7235 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:41:23 dist-test-slave-7kzw krb5kdc[6632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387283, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:41:23.625581 7235 init.cc:377] Logged in from keytab as kudu/127.4.228.195@KRBTEST.COM (short username kudu)
I20260810 18:41:23.627362 7235 webserver.cc:460] Webserver started at http://127.4.228.195:43445/ using document root <none> and password file <none>
I20260810 18:41:23.628119 7235 fs_manager.cc:359] Metadata directory not provided
I20260810 18:41:23.628216 7235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:41:23.628610 7235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:41:23.631940 7235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-2/data/instance:
uuid: "6a774567050643dcaa202b8bc5fd2946"
format_stamp: "Formatted at 2026-08-10 18:41:23 on dist-test-slave-7kzw"
server_key: "c0634ddf50e44586eb04a74b55e4fc99"
server_key_iv: "d2967df699aed44850a101e047a70fda"
server_key_version: "encryptionkey@0"
I20260810 18:41:23.632859 7235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance:
uuid: "6a774567050643dcaa202b8bc5fd2946"
format_stamp: "Formatted at 2026-08-10 18:41:23 on dist-test-slave-7kzw"
server_key: "c0634ddf50e44586eb04a74b55e4fc99"
server_key_iv: "d2967df699aed44850a101e047a70fda"
server_key_version: "encryptionkey@0"
I20260810 18:41:23.639187 7235 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260810 18:41:23.643306 7265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:23.644920 7235 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:41:23.645166 7235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-2/wal
uuid: "6a774567050643dcaa202b8bc5fd2946"
format_stamp: "Formatted at 2026-08-10 18:41:23 on dist-test-slave-7kzw"
server_key: "c0634ddf50e44586eb04a74b55e4fc99"
server_key_iv: "d2967df699aed44850a101e047a70fda"
server_key_version: "encryptionkey@0"
I20260810 18:41:23.645428 7235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:41:23.671362 7235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:41:23.676280 7235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:41:23.676626 7235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:41:23.679741 7235 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:41:23.679863 7235 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:23.680030 7235 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:41:23.680125 7235 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 18:41:23 dist-test-slave-7kzw krb5kdc[6632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387283, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:23.706590 7277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:23.680541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:39239 (local address 127.4.228.254:45135)
0810 18:41:23.681165 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:23.681172 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:23.682940 (+ 1768us) server_negotiation.cc:407] Connection header received
0810 18:41:23.685764 (+ 2824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:23.685771 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:23.685884 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:23.686029 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:23.689041 (+ 3012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:23.690199 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:23.693540 (+ 3341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:23.694152 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:23.694317 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:23.699747 (+ 5430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:23.699753 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:23.699756 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:23.699794 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:23.702816 (+ 3022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:23.703617 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:23.703627 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:23.703629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:23.703811 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:23.704505 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:23.704509 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:23.704516 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:23.704866 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:23.705044 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:23.705857 (+ 813us) server_negotiation.cc:299] Negotiation successful
0810 18:41:23.706111 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":360,"threads_started":1}
I20260810 18:41:23.707865 7278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:23.681402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45135 (local address 127.4.228.195:39239)
0810 18:41:23.682611 (+ 1209us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:23.682765 (+ 154us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:23.685284 (+ 2519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:23.686412 (+ 1128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:23.686427 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:23.687570 (+ 1143us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:23.688775 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:23.688812 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:23.690394 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:23.690406 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:23.693333 (+ 2927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:23.693357 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:23.694716 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:23.694728 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:23.695042 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:23.695999 (+ 957us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:41:23.696037 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:23.699464 (+ 3427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:23.702975 (+ 3511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:23.702986 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:23.703010 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:23.703467 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:23.703934 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:23.703939 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:23.703943 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:23.704273 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:23.705020 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:23.705030 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:23.705459 (+ 429us) client_negotiation.cc:769] Sending connection context
0810 18:41:23.705752 (+ 293us) client_negotiation.cc:240] Negotiation successful
0810 18:41:23.706370 (+ 618us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":821,"thread_start_us":548,"threads_started":1}
I20260810 18:41:23.732352 7277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:23.716104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:39447 (local address 127.4.228.254:45135)
0810 18:41:23.716355 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:23.716361 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:23.717335 (+ 974us) server_negotiation.cc:407] Connection header received
0810 18:41:23.717511 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:23.717521 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:23.717631 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:23.717772 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:23.719004 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:23.719826 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:23.721655 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:23.722123 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:23.722278 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:23.724570 (+ 2292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:23.724576 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:23.724580 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:23.724616 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:23.728770 (+ 4154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:23.729820 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:23.729827 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:23.729831 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:23.729982 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:23.730594 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:23.730602 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:23.730606 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:23.730922 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:23.731121 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:23.731653 (+ 532us) server_negotiation.cc:299] Negotiation successful
0810 18:41:23.731919 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260810 18:41:23.732517 7374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:23.716238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45135 (local address 127.4.228.195:39447)
0810 18:41:23.717195 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:23.717218 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:23.717369 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:23.717821 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:23.717827 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:23.718335 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:23.718815 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:23.718831 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:23.719976 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:23.719984 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:23.721462 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:23.721477 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:23.722354 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:23.722362 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:23.722596 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:23.723481 (+ 885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:41:23.723510 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:23.724351 (+ 841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:23.728935 (+ 4584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:23.728941 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:23.728946 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:23.729608 (+ 662us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:23.730187 (+ 579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:23.730196 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:23.730202 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:23.730428 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:23.731061 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:23.731067 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:23.731349 (+ 282us) client_negotiation.cc:769] Sending connection context
0810 18:41:23.731519 (+ 170us) client_negotiation.cc:240] Negotiation successful
0810 18:41:23.731936 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":788,"thread_start_us":621,"threads_started":1}
I20260810 18:41:23.743645 7235 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.195:38957
I20260810 18:41:23.743666 7443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.195:38957 every 8 connection(s)
I20260810 18:41:23.745042 7235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
I20260810 18:41:23.747462 7445 heartbeater.cc:347] Connected to a master server at 127.4.228.254:45135
I20260810 18:41:23.747759 7445 heartbeater.cc:464] Registering TS with master...
I20260810 18:41:23.748493 7445 heartbeater.cc:511] Master 127.4.228.254:45135 requested a full tablet report, sending...
I20260810 18:41:23.750238 6694 ts_manager.cc:194] Registered new tserver with Master: 6a774567050643dcaa202b8bc5fd2946 (127.4.228.195:38957)
I20260810 18:41:23.751186 6694 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.195@KRBTEST.COM'} at 127.4.228.195:39239
I20260810 18:41:23.752012 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 7235
I20260810 18:41:23.752257 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance
I20260810 18:41:23.752879 5011 external_mini_cluster.cc:1428] Setting key ea4967f57ace6facc12e8d617fced6b3
I20260810 18:41:23.756407 5011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 18:41:23 dist-test-slave-7kzw krb5kdc[6632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387283, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 18:41:23 dist-test-slave-7kzw krb5kdc[6632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387283, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:23.837366 7277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:23.816248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57876 (local address 127.4.228.254:45135)
0810 18:41:23.816487 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:23.816497 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:23.817000 (+ 503us) server_negotiation.cc:407] Connection header received
0810 18:41:23.817298 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:23.817307 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:23.817440 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:23.817573 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:23.819814 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:23.820638 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:23.822971 (+ 2333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:23.823725 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:23.823923 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:23.829704 (+ 5781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:23.829713 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:23.829719 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:23.829775 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:23.833664 (+ 3889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:23.834502 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:23.834510 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:23.834517 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:23.834720 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:23.835320 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:23.835326 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:23.835329 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:23.835783 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:23.836120 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:23.836635 (+ 515us) server_negotiation.cc:299] Negotiation successful
0810 18:41:23.836917 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 18:41:23.843554 6694 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57876:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:41:23.858387 6694 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:41:23.896198 7468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:23.876811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:38957 (local address 127.0.0.1:37198)
0810 18:41:23.880552 (+ 3741us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:23.880581 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:23.880888 (+ 307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:23.882382 (+ 1494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:23.882395 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:23.882444 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:41:23.882973 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:23.882991 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:23.885925 (+ 2934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:23.885935 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:23.889198 (+ 3263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:23.889225 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:23.894630 (+ 5405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:23.894642 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:23.894997 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:23.895091 (+ 94us) client_negotiation.cc:769] Sending connection context
0810 18:41:23.895249 (+ 158us) client_negotiation.cc:240] Negotiation successful
0810 18:41:23.895541 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3531,"mutex_wait_us":1,"thread_start_us":13927,"threads_started":1}
I20260810 18:41:23.897503 7467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:23.877510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37198 (local address 127.4.228.195:38957)
0810 18:41:23.878656 (+ 1146us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:23.878669 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:23.880724 (+ 2055us) server_negotiation.cc:407] Connection header received
0810 18:41:23.881167 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:23.881182 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:23.881586 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:23.881823 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:41:23.883263 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:23.885646 (+ 2383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:23.889481 (+ 3835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:23.893874 (+ 4393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:23.894045 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:23.894389 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:23.896819 (+ 2430us) server_negotiation.cc:299] Negotiation successful
0810 18:41:23.897071 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":862,"thread_start_us":556,"threads_started":1}
I20260810 18:41:23.902307 7463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:23.876401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:33349 (local address 127.0.0.1:51930)
0810 18:41:23.877279 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:23.877351 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:23.877598 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:23.892054 (+ 14456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:23.892058 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:23.892088 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:41:23.892605 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:23.892620 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:23.894456 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:23.894494 (+ 38us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:23.896837 (+ 2343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:23.896855 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:23.901096 (+ 4241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:23.901108 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:23.901319 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:23.901357 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 18:41:23.901439 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 18:41:23.901599 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"mutex_wait_us":6,"thread_start_us":388,"threads_started":1}
I20260810 18:41:23.903164 7464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:23.879544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:45063 (local address 127.0.0.1:47016)
0810 18:41:23.891020 (+ 11476us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:23.891044 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:23.891201 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:23.892290 (+ 1089us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:23.892296 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:23.892330 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:41:23.893672 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:23.893689 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:23.895669 (+ 1980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:23.895675 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:23.897890 (+ 2215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:23.897911 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:23.901653 (+ 3742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:23.901663 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:23.901970 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:23.902027 (+ 57us) client_negotiation.cc:769] Sending connection context
0810 18:41:23.902282 (+ 255us) client_negotiation.cc:240] Negotiation successful
0810 18:41:23.902498 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11332,"mutex_wait_us":900,"thread_start_us":1195,"threads_started":1}
I20260810 18:41:23.903858 7466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:23.877105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51930 (local address 127.4.228.194:33349)
0810 18:41:23.891428 (+ 14323us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:23.891436 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:23.891459 (+ 23us) server_negotiation.cc:407] Connection header received
0810 18:41:23.891540 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:23.891546 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:23.891807 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:23.892663 (+ 856us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:41:23.892881 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:23.894290 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:23.897324 (+ 3034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:23.900701 (+ 3377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:23.902742 (+ 2041us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:23.903063 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:23.903147 (+ 84us) server_negotiation.cc:299] Negotiation successful
0810 18:41:23.903431 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14142,"thread_start_us":14012,"threads_started":1}
I20260810 18:41:23.903618 7342 tablet_service.cc:1467] Processing CreateTablet for tablet 5c1cc99ccba54d08b95f516d753e8995 (DEFAULT_TABLE table=test-table [id=976936a0b3c54beabc50fa4f24a98a06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:41:23.904870 7473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:23.880529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47016 (local address 127.4.228.193:45063)
0810 18:41:23.890245 (+ 9716us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:23.890268 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:23.891373 (+ 1105us) server_negotiation.cc:407] Connection header received
0810 18:41:23.891499 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:23.891508 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:23.891807 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:23.892031 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:41:23.893906 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:23.895446 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:23.898100 (+ 2654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:23.900912 (+ 2812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:23.901091 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:23.901368 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:23.903473 (+ 2105us) server_negotiation.cc:299] Negotiation successful
0810 18:41:23.903721 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9465,"thread_start_us":9743,"threads_started":1}
I20260810 18:41:23.906913 7342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c1cc99ccba54d08b95f516d753e8995. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:23.909192 7121 tablet_service.cc:1467] Processing CreateTablet for tablet 5c1cc99ccba54d08b95f516d753e8995 (DEFAULT_TABLE table=test-table [id=976936a0b3c54beabc50fa4f24a98a06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:41:23.911285 7121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c1cc99ccba54d08b95f516d753e8995. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:23.912120 6909 tablet_service.cc:1467] Processing CreateTablet for tablet 5c1cc99ccba54d08b95f516d753e8995 (DEFAULT_TABLE table=test-table [id=976936a0b3c54beabc50fa4f24a98a06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:41:23.914547 6909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c1cc99ccba54d08b95f516d753e8995. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:23.935297 7474 tablet_bootstrap.cc:492] T 5c1cc99ccba54d08b95f516d753e8995 P 6a774567050643dcaa202b8bc5fd2946: Bootstrap starting.
I20260810 18:41:23.942752 7474 tablet_bootstrap.cc:654] T 5c1cc99ccba54d08b95f516d753e8995 P 6a774567050643dcaa202b8bc5fd2946: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:23.942703 7475 tablet_bootstrap.cc:492] T 5c1cc99ccba54d08b95f516d753e8995 P 54b83db5272749afbdc9a6d659eed5d2: Bootstrap starting.
I20260810 18:41:23.945127 7474 log.cc:824] T 5c1cc99ccba54d08b95f516d753e8995 P 6a774567050643dcaa202b8bc5fd2946: Log is configured to *not* fsync() on all Append() calls
I20260810 18:41:23.947122 7476 tablet_bootstrap.cc:492] T 5c1cc99ccba54d08b95f516d753e8995 P 444dd6af1e194cb9bb17cc9a351457ea: Bootstrap starting.
I20260810 18:41:23.950253 7475 tablet_bootstrap.cc:654] T 5c1cc99ccba54d08b95f516d753e8995 P 54b83db5272749afbdc9a6d659eed5d2: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:23.952514 7475 log.cc:824] T 5c1cc99ccba54d08b95f516d753e8995 P 54b83db5272749afbdc9a6d659eed5d2: Log is configured to *not* fsync() on all Append() calls
I20260810 18:41:23.955720 7476 tablet_bootstrap.cc:654] T 5c1cc99ccba54d08b95f516d753e8995 P 444dd6af1e194cb9bb17cc9a351457ea: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:23.956642 7474 tablet_bootstrap.cc:492] T 5c1cc99ccba54d08b95f516d753e8995 P 6a774567050643dcaa202b8bc5fd2946: No bootstrap required, opened a new log
I20260810 18:41:23.957129 7474 ts_tablet_manager.cc:1397] T 5c1cc99ccba54d08b95f516d753e8995 P 6a774567050643dcaa202b8bc5fd2946: Time spent bootstrapping tablet: real 0.023s user 0.007s sys 0.009s
I20260810 18:41:23.958061 7475 tablet_bootstrap.cc:492] T 5c1cc99ccba54d08b95f516d753e8995 P 54b83db5272749afbdc9a6d659eed5d2: No bootstrap required, opened a new log
I20260810 18:41:23.958570 7475 ts_tablet_manager.cc:1397] T 5c1cc99ccba54d08b95f516d753e8995 P 54b83db5272749afbdc9a6d659eed5d2: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.004s
I20260810 18:41:23.958571 7476 log.cc:824] T 5c1cc99ccba54d08b95f516d753e8995 P 444dd6af1e194cb9bb17cc9a351457ea: Log is configured to *not* fsync() on all Append() calls
I20260810 18:41:23.965727 7476 tablet_bootstrap.cc:492] T 5c1cc99ccba54d08b95f516d753e8995 P 444dd6af1e194cb9bb17cc9a351457ea: No bootstrap required, opened a new log
I20260810 18:41:23.966253 7476 ts_tablet_manager.cc:1397] T 5c1cc99ccba54d08b95f516d753e8995 P 444dd6af1e194cb9bb17cc9a351457ea: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.009s
I20260810 18:41:23.971220 7474 raft_consensus.cc:348] T 5c1cc99ccba54d08b95f516d753e8995 P 6a774567050643dcaa202b8bc5fd2946 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } } peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } }
I20260810 18:41:23.971905 7474 raft_consensus.cc:374] T 5c1cc99ccba54d08b95f516d753e8995 P 6a774567050643dcaa202b8bc5fd2946 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:23.972127 7474 raft_consensus.cc:729] T 5c1cc99ccba54d08b95f516d753e8995 P 6a774567050643dcaa202b8bc5fd2946 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a774567050643dcaa202b8bc5fd2946, State: Initialized, Role: FOLLOWER
I20260810 18:41:23.972966 7474 consensus_queue.cc:260] T 5c1cc99ccba54d08b95f516d753e8995 P 6a774567050643dcaa202b8bc5fd2946 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } } peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } }
I20260810 18:41:23.975046 7445 heartbeater.cc:503] Master 127.4.228.254:45135 was elected leader, sending a full tablet report...
I20260810 18:41:23.975926 7474 ts_tablet_manager.cc:1428] T 5c1cc99ccba54d08b95f516d753e8995 P 6a774567050643dcaa202b8bc5fd2946: Time spent starting tablet: real 0.019s user 0.013s sys 0.006s
I20260810 18:41:23.976823 7475 raft_consensus.cc:348] T 5c1cc99ccba54d08b95f516d753e8995 P 54b83db5272749afbdc9a6d659eed5d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } } peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } }
I20260810 18:41:23.977573 7475 raft_consensus.cc:374] T 5c1cc99ccba54d08b95f516d753e8995 P 54b83db5272749afbdc9a6d659eed5d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:23.977772 7475 raft_consensus.cc:729] T 5c1cc99ccba54d08b95f516d753e8995 P 54b83db5272749afbdc9a6d659eed5d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54b83db5272749afbdc9a6d659eed5d2, State: Initialized, Role: FOLLOWER
I20260810 18:41:23.978703 7475 consensus_queue.cc:260] T 5c1cc99ccba54d08b95f516d753e8995 P 54b83db5272749afbdc9a6d659eed5d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } } peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } }
I20260810 18:41:23.980551 7227 heartbeater.cc:503] Master 127.4.228.254:45135 was elected leader, sending a full tablet report...
I20260810 18:41:23.981662 7475 ts_tablet_manager.cc:1428] T 5c1cc99ccba54d08b95f516d753e8995 P 54b83db5272749afbdc9a6d659eed5d2: Time spent starting tablet: real 0.023s user 0.017s sys 0.005s
I20260810 18:41:23.986943 7476 raft_consensus.cc:348] T 5c1cc99ccba54d08b95f516d753e8995 P 444dd6af1e194cb9bb17cc9a351457ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } } peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } }
I20260810 18:41:23.987520 7476 raft_consensus.cc:374] T 5c1cc99ccba54d08b95f516d753e8995 P 444dd6af1e194cb9bb17cc9a351457ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:23.987646 7476 raft_consensus.cc:729] T 5c1cc99ccba54d08b95f516d753e8995 P 444dd6af1e194cb9bb17cc9a351457ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444dd6af1e194cb9bb17cc9a351457ea, State: Initialized, Role: FOLLOWER
I20260810 18:41:23.988330 7476 consensus_queue.cc:260] T 5c1cc99ccba54d08b95f516d753e8995 P 444dd6af1e194cb9bb17cc9a351457ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } } peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } }
I20260810 18:41:23.990733 7476 ts_tablet_manager.cc:1428] T 5c1cc99ccba54d08b95f516d753e8995 P 444dd6af1e194cb9bb17cc9a351457ea: Time spent starting tablet: real 0.024s user 0.020s sys 0.006s
W20260810 18:41:23.996990 7446 tablet.cc:2369] T 5c1cc99ccba54d08b95f516d753e8995 P 6a774567050643dcaa202b8bc5fd2946: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:41:24.012557 7482 raft_consensus.cc:482] T 5c1cc99ccba54d08b95f516d753e8995 P 6a774567050643dcaa202b8bc5fd2946 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:41:24.013230 7482 raft_consensus.cc:504] T 5c1cc99ccba54d08b95f516d753e8995 P 6a774567050643dcaa202b8bc5fd2946 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } } peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } }
I20260810 18:41:24.015913 7482 leader_election.cc:288] T 5c1cc99ccba54d08b95f516d753e8995 P 6a774567050643dcaa202b8bc5fd2946 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 54b83db5272749afbdc9a6d659eed5d2 (127.4.228.194:33349), 444dd6af1e194cb9bb17cc9a351457ea (127.4.228.193:45063)
W20260810 18:41:24.022698 7012 tablet.cc:2369] T 5c1cc99ccba54d08b95f516d753e8995 P 444dd6af1e194cb9bb17cc9a351457ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:41:24.027311 7278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:24.016208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:45063 (local address 127.4.228.195:42631)
0810 18:41:24.017085 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:24.017111 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:24.017299 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:24.017856 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:24.017872 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:24.017919 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:41:24.018371 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:24.018388 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:24.020653 (+ 2265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:24.020658 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:24.023805 (+ 3147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:24.023825 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:24.026143 (+ 2318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:24.026148 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:24.026673 (+ 525us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:24.026704 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 18:41:24.026806 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 18:41:24.026919 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"mutex_wait_us":1}
I20260810 18:41:24.027527 7473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:24.016455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:42631 (local address 127.4.228.193:45063)
0810 18:41:24.016855 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:24.016863 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:24.017268 (+ 405us) server_negotiation.cc:407] Connection header received
0810 18:41:24.017520 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:24.017528 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:24.017647 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:24.017792 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:41:24.018699 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:24.020392 (+ 1693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:24.024099 (+ 3707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:24.025891 (+ 1792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:24.026140 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:24.026245 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:24.026884 (+ 639us) server_negotiation.cc:299] Negotiation successful
0810 18:41:24.027063 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260810 18:41:24.028534 7466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:24.016753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:41833 (local address 127.4.228.194:33349)
0810 18:41:24.017033 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:24.017040 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:24.018591 (+ 1551us) server_negotiation.cc:407] Connection header received
0810 18:41:24.018777 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:24.018783 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:24.018948 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:24.019152 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:41:24.020048 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:24.021368 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:24.025051 (+ 3683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:24.027038 (+ 1987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:24.027287 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:24.027370 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:24.027907 (+ 537us) server_negotiation.cc:299] Negotiation successful
0810 18:41:24.028064 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 18:41:24.028785 7486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:24.016692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:33349 (local address 127.4.228.195:41833)
0810 18:41:24.018377 (+ 1685us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:24.018421 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:24.018628 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:24.019187 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:24.019191 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:24.019214 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:41:24.019847 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:24.019857 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:24.021561 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:24.021567 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:24.024690 (+ 3123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:24.024704 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:24.027438 (+ 2734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:24.027446 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:24.027680 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:24.027758 (+ 78us) client_negotiation.cc:769] Sending connection context
0810 18:41:24.027876 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 18:41:24.028061 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1432,"mutex_wait_us":6,"thread_start_us":1050,"threads_started":1}
I20260810 18:41:24.029031 6942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c1cc99ccba54d08b95f516d753e8995" candidate_uuid: "6a774567050643dcaa202b8bc5fd2946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444dd6af1e194cb9bb17cc9a351457ea" is_pre_election: true
I20260810 18:41:24.029902 6942 raft_consensus.cc:2454] T 5c1cc99ccba54d08b95f516d753e8995 P 444dd6af1e194cb9bb17cc9a351457ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a774567050643dcaa202b8bc5fd2946 in term 0.
I20260810 18:41:24.031366 7269 leader_election.cc:302] T 5c1cc99ccba54d08b95f516d753e8995 P 6a774567050643dcaa202b8bc5fd2946 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444dd6af1e194cb9bb17cc9a351457ea, 6a774567050643dcaa202b8bc5fd2946; no voters:
I20260810 18:41:24.031450 7151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c1cc99ccba54d08b95f516d753e8995" candidate_uuid: "6a774567050643dcaa202b8bc5fd2946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54b83db5272749afbdc9a6d659eed5d2" is_pre_election: true
I20260810 18:41:24.032272 7482 raft_consensus.cc:2781] T 5c1cc99ccba54d08b95f516d753e8995 P 6a774567050643dcaa202b8bc5fd2946 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:41:24.032531 7482 raft_consensus.cc:482] T 5c1cc99ccba54d08b95f516d753e8995 P 6a774567050643dcaa202b8bc5fd2946 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:41:24.032490 7151 raft_consensus.cc:2454] T 5c1cc99ccba54d08b95f516d753e8995 P 54b83db5272749afbdc9a6d659eed5d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a774567050643dcaa202b8bc5fd2946 in term 0.
I20260810 18:41:24.032757 7482 raft_consensus.cc:3037] T 5c1cc99ccba54d08b95f516d753e8995 P 6a774567050643dcaa202b8bc5fd2946 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:24.036660 7482 raft_consensus.cc:504] T 5c1cc99ccba54d08b95f516d753e8995 P 6a774567050643dcaa202b8bc5fd2946 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } } peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } }
I20260810 18:41:24.037912 7482 leader_election.cc:288] T 5c1cc99ccba54d08b95f516d753e8995 P 6a774567050643dcaa202b8bc5fd2946 [CANDIDATE]: Term 1 election: Requested vote from peers 54b83db5272749afbdc9a6d659eed5d2 (127.4.228.194:33349), 444dd6af1e194cb9bb17cc9a351457ea (127.4.228.193:45063)
I20260810 18:41:24.038403 7151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c1cc99ccba54d08b95f516d753e8995" candidate_uuid: "6a774567050643dcaa202b8bc5fd2946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54b83db5272749afbdc9a6d659eed5d2"
I20260810 18:41:24.039232 7151 raft_consensus.cc:3037] T 5c1cc99ccba54d08b95f516d753e8995 P 54b83db5272749afbdc9a6d659eed5d2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:24.039301 6942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c1cc99ccba54d08b95f516d753e8995" candidate_uuid: "6a774567050643dcaa202b8bc5fd2946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444dd6af1e194cb9bb17cc9a351457ea"
I20260810 18:41:24.039856 6942 raft_consensus.cc:3037] T 5c1cc99ccba54d08b95f516d753e8995 P 444dd6af1e194cb9bb17cc9a351457ea [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:24.045746 6942 raft_consensus.cc:2454] T 5c1cc99ccba54d08b95f516d753e8995 P 444dd6af1e194cb9bb17cc9a351457ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a774567050643dcaa202b8bc5fd2946 in term 1.
I20260810 18:41:24.045742 7151 raft_consensus.cc:2454] T 5c1cc99ccba54d08b95f516d753e8995 P 54b83db5272749afbdc9a6d659eed5d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a774567050643dcaa202b8bc5fd2946 in term 1.
I20260810 18:41:24.046579 7267 leader_election.cc:302] T 5c1cc99ccba54d08b95f516d753e8995 P 6a774567050643dcaa202b8bc5fd2946 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54b83db5272749afbdc9a6d659eed5d2, 6a774567050643dcaa202b8bc5fd2946; no voters:
I20260810 18:41:24.047159 7482 raft_consensus.cc:2781] T 5c1cc99ccba54d08b95f516d753e8995 P 6a774567050643dcaa202b8bc5fd2946 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:41:24.048259 7482 raft_consensus.cc:686] T 5c1cc99ccba54d08b95f516d753e8995 P 6a774567050643dcaa202b8bc5fd2946 [term 1 LEADER]: Becoming Leader. State: Replica: 6a774567050643dcaa202b8bc5fd2946, State: Running, Role: LEADER
I20260810 18:41:24.049106 7482 consensus_queue.cc:237] T 5c1cc99ccba54d08b95f516d753e8995 P 6a774567050643dcaa202b8bc5fd2946 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } } peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } }
I20260810 18:41:24.055985 6693 catalog_manager.cc:5495] T 5c1cc99ccba54d08b95f516d753e8995 P 6a774567050643dcaa202b8bc5fd2946 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a774567050643dcaa202b8bc5fd2946 (127.4.228.195). New cstate: current_term: 1 leader_uuid: "6a774567050643dcaa202b8bc5fd2946" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } health_report { overall_health: UNKNOWN } } }
I20260810 18:41:24.106835 7467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:24.099006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37210 (local address 127.4.228.195:38957)
0810 18:41:24.099363 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:24.099372 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:24.099523 (+ 151us) server_negotiation.cc:407] Connection header received
0810 18:41:24.100002 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:24.100006 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:24.100135 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:24.100295 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:41:24.100805 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:24.101809 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:24.103607 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:24.104230 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:24.104390 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:24.105122 (+ 732us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:41:24.105820 (+ 698us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:41:24.105998 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:24.106178 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 18:41:24.106272 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260810 18:41:24.119689 6942 raft_consensus.cc:1261] T 5c1cc99ccba54d08b95f516d753e8995 P 444dd6af1e194cb9bb17cc9a351457ea [term 1 FOLLOWER]: Refusing update from remote peer 6a774567050643dcaa202b8bc5fd2946: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:41:24.121397 7151 raft_consensus.cc:1261] T 5c1cc99ccba54d08b95f516d753e8995 P 54b83db5272749afbdc9a6d659eed5d2 [term 1 FOLLOWER]: Refusing update from remote peer 6a774567050643dcaa202b8bc5fd2946: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:41:24.121125 7489 consensus_queue.cc:1035] T 5c1cc99ccba54d08b95f516d753e8995 P 6a774567050643dcaa202b8bc5fd2946 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:41:24.122687 7489 consensus_queue.cc:1035] T 5c1cc99ccba54d08b95f516d753e8995 P 6a774567050643dcaa202b8bc5fd2946 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:41:24.140877 7493 mvcc.cc:205] Tried to move back new op lower bound from 7317042315738091520 to 7317042315470888960. Current Snapshot: MvccSnapshot[applied={T|T < 7317042315738091520}]
I20260810 18:41:24.181471 6693 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:41:24.182091 6693 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:41:24.189105 6693 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5a659e0d5d194cfa93aa8c8d12e9ccf3: Sending DeleteTablet for 3 replicas of tablet 5c1cc99ccba54d08b95f516d753e8995
I20260810 18:41:24.191195 7121 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c1cc99ccba54d08b95f516d753e8995 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:41:24 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:51930
I20260810 18:41:24.191334 7342 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c1cc99ccba54d08b95f516d753e8995 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:41:24 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:37198
I20260810 18:41:24.191709 6909 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c1cc99ccba54d08b95f516d753e8995 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:41:24 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:47016
I20260810 18:41:24.194904 6694 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57876:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:41:24.196237 6694 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:41:24.204082 7511 tablet_replica.cc:330] T 5c1cc99ccba54d08b95f516d753e8995 P 444dd6af1e194cb9bb17cc9a351457ea: stopping tablet replica
I20260810 18:41:24.204159 7510 tablet_replica.cc:330] T 5c1cc99ccba54d08b95f516d753e8995 P 54b83db5272749afbdc9a6d659eed5d2: stopping tablet replica
I20260810 18:41:24.204969 7510 raft_consensus.cc:2229] T 5c1cc99ccba54d08b95f516d753e8995 P 54b83db5272749afbdc9a6d659eed5d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:41:24.205041 7511 raft_consensus.cc:2229] T 5c1cc99ccba54d08b95f516d753e8995 P 444dd6af1e194cb9bb17cc9a351457ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:41:24.206039 7511 raft_consensus.cc:2258] T 5c1cc99ccba54d08b95f516d753e8995 P 444dd6af1e194cb9bb17cc9a351457ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:41:24.206039 7510 raft_consensus.cc:2258] T 5c1cc99ccba54d08b95f516d753e8995 P 54b83db5272749afbdc9a6d659eed5d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:41:24.206048 7509 tablet_replica.cc:330] T 5c1cc99ccba54d08b95f516d753e8995 P 6a774567050643dcaa202b8bc5fd2946: stopping tablet replica
I20260810 18:41:24.207238 7509 raft_consensus.cc:2229] T 5c1cc99ccba54d08b95f516d753e8995 P 6a774567050643dcaa202b8bc5fd2946 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:41:24.208808 7510 ts_tablet_manager.cc:1905] T 5c1cc99ccba54d08b95f516d753e8995 P 54b83db5272749afbdc9a6d659eed5d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:41:24.210541 7511 ts_tablet_manager.cc:1905] T 5c1cc99ccba54d08b95f516d753e8995 P 444dd6af1e194cb9bb17cc9a351457ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:41:24.210783 7121 tablet_service.cc:1467] Processing CreateTablet for tablet 079466a76e5648b88dfeb29954275e39 (DEFAULT_TABLE table=test-table [id=9cf802616fb74d6eb159bb8819b47d34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:41:24.210829 6909 tablet_service.cc:1467] Processing CreateTablet for tablet 079466a76e5648b88dfeb29954275e39 (DEFAULT_TABLE table=test-table [id=9cf802616fb74d6eb159bb8819b47d34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:41:24.211426 6909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 079466a76e5648b88dfeb29954275e39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:24.211550 7121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 079466a76e5648b88dfeb29954275e39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:24.211102 7509 raft_consensus.cc:2258] T 5c1cc99ccba54d08b95f516d753e8995 P 6a774567050643dcaa202b8bc5fd2946 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:41:24.220993 7510 ts_tablet_manager.cc:1918] T 5c1cc99ccba54d08b95f516d753e8995 P 54b83db5272749afbdc9a6d659eed5d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:41:24.221481 7510 log.cc:1196] T 5c1cc99ccba54d08b95f516d753e8995 P 54b83db5272749afbdc9a6d659eed5d2: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-1/wal/wals/5c1cc99ccba54d08b95f516d753e8995
I20260810 18:41:24.221688 7342 tablet_service.cc:1467] Processing CreateTablet for tablet 079466a76e5648b88dfeb29954275e39 (DEFAULT_TABLE table=test-table [id=9cf802616fb74d6eb159bb8819b47d34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:41:24.222564 7510 ts_tablet_manager.cc:1939] T 5c1cc99ccba54d08b95f516d753e8995 P 54b83db5272749afbdc9a6d659eed5d2: Deleting consensus metadata
I20260810 18:41:24.223126 7509 ts_tablet_manager.cc:1905] T 5c1cc99ccba54d08b95f516d753e8995 P 6a774567050643dcaa202b8bc5fd2946: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:41:24.223443 7342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 079466a76e5648b88dfeb29954275e39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:24.223984 7475 tablet_bootstrap.cc:492] T 079466a76e5648b88dfeb29954275e39 P 54b83db5272749afbdc9a6d659eed5d2: Bootstrap starting.
I20260810 18:41:24.226616 7475 tablet_bootstrap.cc:654] T 079466a76e5648b88dfeb29954275e39 P 54b83db5272749afbdc9a6d659eed5d2: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:24.227175 6670 catalog_manager.cc:4841] TS 54b83db5272749afbdc9a6d659eed5d2 (127.4.228.194:33349): tablet 5c1cc99ccba54d08b95f516d753e8995 (table test-table [id=976936a0b3c54beabc50fa4f24a98a06]) successfully deleted
I20260810 18:41:24.227533 7476 tablet_bootstrap.cc:492] T 079466a76e5648b88dfeb29954275e39 P 444dd6af1e194cb9bb17cc9a351457ea: Bootstrap starting.
I20260810 18:41:24.230170 7476 tablet_bootstrap.cc:654] T 079466a76e5648b88dfeb29954275e39 P 444dd6af1e194cb9bb17cc9a351457ea: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:24.239689 7511 ts_tablet_manager.cc:1918] T 5c1cc99ccba54d08b95f516d753e8995 P 444dd6af1e194cb9bb17cc9a351457ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:41:24.240015 7511 log.cc:1196] T 5c1cc99ccba54d08b95f516d753e8995 P 444dd6af1e194cb9bb17cc9a351457ea: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-0/wal/wals/5c1cc99ccba54d08b95f516d753e8995
I20260810 18:41:24.241091 7511 ts_tablet_manager.cc:1939] T 5c1cc99ccba54d08b95f516d753e8995 P 444dd6af1e194cb9bb17cc9a351457ea: Deleting consensus metadata
I20260810 18:41:24.244630 6671 catalog_manager.cc:4841] TS 444dd6af1e194cb9bb17cc9a351457ea (127.4.228.193:45063): tablet 5c1cc99ccba54d08b95f516d753e8995 (table test-table [id=976936a0b3c54beabc50fa4f24a98a06]) successfully deleted
I20260810 18:41:24.244740 7509 ts_tablet_manager.cc:1918] T 5c1cc99ccba54d08b95f516d753e8995 P 6a774567050643dcaa202b8bc5fd2946: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:41:24.245044 7509 log.cc:1196] T 5c1cc99ccba54d08b95f516d753e8995 P 6a774567050643dcaa202b8bc5fd2946: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-2/wal/wals/5c1cc99ccba54d08b95f516d753e8995
I20260810 18:41:24.245244 7475 tablet_bootstrap.cc:492] T 079466a76e5648b88dfeb29954275e39 P 54b83db5272749afbdc9a6d659eed5d2: No bootstrap required, opened a new log
I20260810 18:41:24.245504 7475 ts_tablet_manager.cc:1397] T 079466a76e5648b88dfeb29954275e39 P 54b83db5272749afbdc9a6d659eed5d2: Time spent bootstrapping tablet: real 0.022s user 0.006s sys 0.005s
I20260810 18:41:24.245550 7476 tablet_bootstrap.cc:492] T 079466a76e5648b88dfeb29954275e39 P 444dd6af1e194cb9bb17cc9a351457ea: No bootstrap required, opened a new log
I20260810 18:41:24.245762 7509 ts_tablet_manager.cc:1939] T 5c1cc99ccba54d08b95f516d753e8995 P 6a774567050643dcaa202b8bc5fd2946: Deleting consensus metadata
I20260810 18:41:24.246058 7476 ts_tablet_manager.cc:1397] T 079466a76e5648b88dfeb29954275e39 P 444dd6af1e194cb9bb17cc9a351457ea: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.015s
I20260810 18:41:24.247140 7475 raft_consensus.cc:348] T 079466a76e5648b88dfeb29954275e39 P 54b83db5272749afbdc9a6d659eed5d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } } peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } }
I20260810 18:41:24.247529 7475 raft_consensus.cc:374] T 079466a76e5648b88dfeb29954275e39 P 54b83db5272749afbdc9a6d659eed5d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:24.247763 7475 raft_consensus.cc:729] T 079466a76e5648b88dfeb29954275e39 P 54b83db5272749afbdc9a6d659eed5d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54b83db5272749afbdc9a6d659eed5d2, State: Initialized, Role: FOLLOWER
I20260810 18:41:24.247985 6674 catalog_manager.cc:4841] TS 6a774567050643dcaa202b8bc5fd2946 (127.4.228.195:38957): tablet 5c1cc99ccba54d08b95f516d753e8995 (table test-table [id=976936a0b3c54beabc50fa4f24a98a06]) successfully deleted
I20260810 18:41:24.248016 7476 raft_consensus.cc:348] T 079466a76e5648b88dfeb29954275e39 P 444dd6af1e194cb9bb17cc9a351457ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } } peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } }
I20260810 18:41:24.248303 7475 consensus_queue.cc:260] T 079466a76e5648b88dfeb29954275e39 P 54b83db5272749afbdc9a6d659eed5d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } } peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } }
I20260810 18:41:24.248553 7476 raft_consensus.cc:374] T 079466a76e5648b88dfeb29954275e39 P 444dd6af1e194cb9bb17cc9a351457ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:24.249459 7474 tablet_bootstrap.cc:492] T 079466a76e5648b88dfeb29954275e39 P 6a774567050643dcaa202b8bc5fd2946: Bootstrap starting.
I20260810 18:41:24.250280 7476 raft_consensus.cc:729] T 079466a76e5648b88dfeb29954275e39 P 444dd6af1e194cb9bb17cc9a351457ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444dd6af1e194cb9bb17cc9a351457ea, State: Initialized, Role: FOLLOWER
I20260810 18:41:24.250886 7476 consensus_queue.cc:260] T 079466a76e5648b88dfeb29954275e39 P 444dd6af1e194cb9bb17cc9a351457ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } } peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } }
I20260810 18:41:24.251947 7476 ts_tablet_manager.cc:1428] T 079466a76e5648b88dfeb29954275e39 P 444dd6af1e194cb9bb17cc9a351457ea: Time spent starting tablet: real 0.006s user 0.002s sys 0.001s
I20260810 18:41:24.252686 7475 ts_tablet_manager.cc:1428] T 079466a76e5648b88dfeb29954275e39 P 54b83db5272749afbdc9a6d659eed5d2: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20260810 18:41:24.254311 7474 tablet_bootstrap.cc:654] T 079466a76e5648b88dfeb29954275e39 P 6a774567050643dcaa202b8bc5fd2946: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:24.259532 7474 tablet_bootstrap.cc:492] T 079466a76e5648b88dfeb29954275e39 P 6a774567050643dcaa202b8bc5fd2946: No bootstrap required, opened a new log
I20260810 18:41:24.259816 7474 ts_tablet_manager.cc:1397] T 079466a76e5648b88dfeb29954275e39 P 6a774567050643dcaa202b8bc5fd2946: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260810 18:41:24.261529 7474 raft_consensus.cc:348] T 079466a76e5648b88dfeb29954275e39 P 6a774567050643dcaa202b8bc5fd2946 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } } peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } }
I20260810 18:41:24.261796 7474 raft_consensus.cc:374] T 079466a76e5648b88dfeb29954275e39 P 6a774567050643dcaa202b8bc5fd2946 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:24.261994 7474 raft_consensus.cc:729] T 079466a76e5648b88dfeb29954275e39 P 6a774567050643dcaa202b8bc5fd2946 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a774567050643dcaa202b8bc5fd2946, State: Initialized, Role: FOLLOWER
I20260810 18:41:24.262356 7474 consensus_queue.cc:260] T 079466a76e5648b88dfeb29954275e39 P 6a774567050643dcaa202b8bc5fd2946 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } } peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } }
I20260810 18:41:24.263236 7474 ts_tablet_manager.cc:1428] T 079466a76e5648b88dfeb29954275e39 P 6a774567050643dcaa202b8bc5fd2946: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
W20260810 18:41:24.404050 7228 tablet.cc:2369] T 079466a76e5648b88dfeb29954275e39 P 54b83db5272749afbdc9a6d659eed5d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:41:24.528291 7502 raft_consensus.cc:482] T 079466a76e5648b88dfeb29954275e39 P 6a774567050643dcaa202b8bc5fd2946 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:41:24.528575 7502 raft_consensus.cc:504] T 079466a76e5648b88dfeb29954275e39 P 6a774567050643dcaa202b8bc5fd2946 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } } peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } }
I20260810 18:41:24.529698 7502 leader_election.cc:288] T 079466a76e5648b88dfeb29954275e39 P 6a774567050643dcaa202b8bc5fd2946 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444dd6af1e194cb9bb17cc9a351457ea (127.4.228.193:45063), 54b83db5272749afbdc9a6d659eed5d2 (127.4.228.194:33349)
I20260810 18:41:24.530211 6942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "079466a76e5648b88dfeb29954275e39" candidate_uuid: "6a774567050643dcaa202b8bc5fd2946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444dd6af1e194cb9bb17cc9a351457ea" is_pre_election: true
I20260810 18:41:24.530450 6942 raft_consensus.cc:2454] T 079466a76e5648b88dfeb29954275e39 P 444dd6af1e194cb9bb17cc9a351457ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a774567050643dcaa202b8bc5fd2946 in term 0.
I20260810 18:41:24.530902 7151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "079466a76e5648b88dfeb29954275e39" candidate_uuid: "6a774567050643dcaa202b8bc5fd2946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54b83db5272749afbdc9a6d659eed5d2" is_pre_election: true
I20260810 18:41:24.531050 7269 leader_election.cc:302] T 079466a76e5648b88dfeb29954275e39 P 6a774567050643dcaa202b8bc5fd2946 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444dd6af1e194cb9bb17cc9a351457ea, 6a774567050643dcaa202b8bc5fd2946; no voters:
I20260810 18:41:24.531306 7151 raft_consensus.cc:2454] T 079466a76e5648b88dfeb29954275e39 P 54b83db5272749afbdc9a6d659eed5d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a774567050643dcaa202b8bc5fd2946 in term 0.
I20260810 18:41:24.531486 7502 raft_consensus.cc:2781] T 079466a76e5648b88dfeb29954275e39 P 6a774567050643dcaa202b8bc5fd2946 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:41:24.531603 7502 raft_consensus.cc:482] T 079466a76e5648b88dfeb29954275e39 P 6a774567050643dcaa202b8bc5fd2946 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:41:24.531742 7502 raft_consensus.cc:3037] T 079466a76e5648b88dfeb29954275e39 P 6a774567050643dcaa202b8bc5fd2946 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:24.535938 7502 raft_consensus.cc:504] T 079466a76e5648b88dfeb29954275e39 P 6a774567050643dcaa202b8bc5fd2946 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } } peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } }
I20260810 18:41:24.536796 7502 leader_election.cc:288] T 079466a76e5648b88dfeb29954275e39 P 6a774567050643dcaa202b8bc5fd2946 [CANDIDATE]: Term 1 election: Requested vote from peers 444dd6af1e194cb9bb17cc9a351457ea (127.4.228.193:45063), 54b83db5272749afbdc9a6d659eed5d2 (127.4.228.194:33349)
I20260810 18:41:24.537776 7151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "079466a76e5648b88dfeb29954275e39" candidate_uuid: "6a774567050643dcaa202b8bc5fd2946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54b83db5272749afbdc9a6d659eed5d2"
I20260810 18:41:24.537801 6942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "079466a76e5648b88dfeb29954275e39" candidate_uuid: "6a774567050643dcaa202b8bc5fd2946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444dd6af1e194cb9bb17cc9a351457ea"
I20260810 18:41:24.538102 7151 raft_consensus.cc:3037] T 079466a76e5648b88dfeb29954275e39 P 54b83db5272749afbdc9a6d659eed5d2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:24.538280 6942 raft_consensus.cc:3037] T 079466a76e5648b88dfeb29954275e39 P 444dd6af1e194cb9bb17cc9a351457ea [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:24.542258 7151 raft_consensus.cc:2454] T 079466a76e5648b88dfeb29954275e39 P 54b83db5272749afbdc9a6d659eed5d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a774567050643dcaa202b8bc5fd2946 in term 1.
I20260810 18:41:24.542883 7267 leader_election.cc:302] T 079466a76e5648b88dfeb29954275e39 P 6a774567050643dcaa202b8bc5fd2946 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54b83db5272749afbdc9a6d659eed5d2, 6a774567050643dcaa202b8bc5fd2946; no voters:
I20260810 18:41:24.543614 7502 raft_consensus.cc:2781] T 079466a76e5648b88dfeb29954275e39 P 6a774567050643dcaa202b8bc5fd2946 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:41:24.543882 7502 raft_consensus.cc:686] T 079466a76e5648b88dfeb29954275e39 P 6a774567050643dcaa202b8bc5fd2946 [term 1 LEADER]: Becoming Leader. State: Replica: 6a774567050643dcaa202b8bc5fd2946, State: Running, Role: LEADER
I20260810 18:41:24.543989 6942 raft_consensus.cc:2454] T 079466a76e5648b88dfeb29954275e39 P 444dd6af1e194cb9bb17cc9a351457ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a774567050643dcaa202b8bc5fd2946 in term 1.
I20260810 18:41:24.544450 7502 consensus_queue.cc:237] T 079466a76e5648b88dfeb29954275e39 P 6a774567050643dcaa202b8bc5fd2946 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } } peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } }
I20260810 18:41:24.548827 6694 catalog_manager.cc:5495] T 079466a76e5648b88dfeb29954275e39 P 6a774567050643dcaa202b8bc5fd2946 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a774567050643dcaa202b8bc5fd2946 (127.4.228.195). New cstate: current_term: 1 leader_uuid: "6a774567050643dcaa202b8bc5fd2946" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } health_report { overall_health: HEALTHY } } }
Aug 10 18:41:24 dist-test-slave-7kzw krb5kdc[6632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387281, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:24.644670 7525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:24.621685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57882 (local address 127.4.228.254:45135)
0810 18:41:24.622894 (+ 1209us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:24.622898 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:24.623705 (+ 807us) server_negotiation.cc:407] Connection header received
0810 18:41:24.624139 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:24.624143 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:24.624250 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:24.624372 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:24.626140 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:24.627216 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:24.628997 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:24.629670 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:24.629807 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:24.635456 (+ 5649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:24.635462 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:24.635466 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:24.635500 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:24.640285 (+ 4785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:24.641888 (+ 1603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:24.641894 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:24.641896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:24.642083 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:24.642796 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:24.642800 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:24.642802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:24.643276 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:24.643451 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:24.643969 (+ 518us) server_negotiation.cc:299] Negotiation successful
0810 18:41:24.644309 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":130,"server-negotiator.queue_time_us":1041,"thread_start_us":838,"threads_started":1}
I20260810 18:41:24.644677 7527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:24.621815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45135 (local address 127.0.0.1:57882)
0810 18:41:24.623575 (+ 1760us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:24.623593 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:24.623928 (+ 335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:24.624403 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:24.624407 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:24.625533 (+ 1126us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:24.625930 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:24.625942 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:24.627419 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:24.627424 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:24.628835 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:24.628850 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:24.629894 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:24.629902 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:24.630166 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:24.631224 (+ 1058us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:41:24.631242 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:24.635154 (+ 3912us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:24.640564 (+ 5410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:24.640587 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:24.640598 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:24.641722 (+ 1124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:24.642254 (+ 532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:24.642260 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:24.642264 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:24.642599 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:24.643438 (+ 839us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:24.643448 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:24.643631 (+ 183us) client_negotiation.cc:769] Sending connection context
0810 18:41:24.643832 (+ 201us) client_negotiation.cc:240] Negotiation successful
0810 18:41:24.644173 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1615,"thread_start_us":1394,"threads_started":1}
I20260810 18:41:24.652314 6694 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:57882:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 18:41:24.653985 6694 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:41:24.663563 7342 tablet_service.cc:1467] Processing CreateTablet for tablet df09f7d1d4064bc89b78728d3f773585 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=91a81db0df22406aa1e1ea0a33b64f4b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 18:41:24.663494 6909 tablet_service.cc:1467] Processing CreateTablet for tablet df09f7d1d4064bc89b78728d3f773585 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=91a81db0df22406aa1e1ea0a33b64f4b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 18:41:24.664139 7342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df09f7d1d4064bc89b78728d3f773585. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:24.664273 6909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df09f7d1d4064bc89b78728d3f773585. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:24.668848 7121 tablet_service.cc:1467] Processing CreateTablet for tablet df09f7d1d4064bc89b78728d3f773585 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=91a81db0df22406aa1e1ea0a33b64f4b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 18:41:24.669689 7121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df09f7d1d4064bc89b78728d3f773585. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:24.675993 7476 tablet_bootstrap.cc:492] T df09f7d1d4064bc89b78728d3f773585 P 444dd6af1e194cb9bb17cc9a351457ea: Bootstrap starting.
I20260810 18:41:24.679055 7476 tablet_bootstrap.cc:654] T df09f7d1d4064bc89b78728d3f773585 P 444dd6af1e194cb9bb17cc9a351457ea: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:24.679330 7474 tablet_bootstrap.cc:492] T df09f7d1d4064bc89b78728d3f773585 P 6a774567050643dcaa202b8bc5fd2946: Bootstrap starting.
I20260810 18:41:24.682894 7474 tablet_bootstrap.cc:654] T df09f7d1d4064bc89b78728d3f773585 P 6a774567050643dcaa202b8bc5fd2946: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:24.685105 7475 tablet_bootstrap.cc:492] T df09f7d1d4064bc89b78728d3f773585 P 54b83db5272749afbdc9a6d659eed5d2: Bootstrap starting.
I20260810 18:41:24.688637 7475 tablet_bootstrap.cc:654] T df09f7d1d4064bc89b78728d3f773585 P 54b83db5272749afbdc9a6d659eed5d2: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:24.691033 7476 tablet_bootstrap.cc:492] T df09f7d1d4064bc89b78728d3f773585 P 444dd6af1e194cb9bb17cc9a351457ea: No bootstrap required, opened a new log
I20260810 18:41:24.691713 7476 ts_tablet_manager.cc:1397] T df09f7d1d4064bc89b78728d3f773585 P 444dd6af1e194cb9bb17cc9a351457ea: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.001s
I20260810 18:41:24.693302 7476 raft_consensus.cc:348] T df09f7d1d4064bc89b78728d3f773585 P 444dd6af1e194cb9bb17cc9a351457ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } } peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } }
I20260810 18:41:24.693813 7474 tablet_bootstrap.cc:492] T df09f7d1d4064bc89b78728d3f773585 P 6a774567050643dcaa202b8bc5fd2946: No bootstrap required, opened a new log
I20260810 18:41:24.693773 7476 raft_consensus.cc:374] T df09f7d1d4064bc89b78728d3f773585 P 444dd6af1e194cb9bb17cc9a351457ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:24.693933 7476 raft_consensus.cc:729] T df09f7d1d4064bc89b78728d3f773585 P 444dd6af1e194cb9bb17cc9a351457ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444dd6af1e194cb9bb17cc9a351457ea, State: Initialized, Role: FOLLOWER
I20260810 18:41:24.694105 7474 ts_tablet_manager.cc:1397] T df09f7d1d4064bc89b78728d3f773585 P 6a774567050643dcaa202b8bc5fd2946: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.009s
I20260810 18:41:24.694597 7476 consensus_queue.cc:260] T df09f7d1d4064bc89b78728d3f773585 P 444dd6af1e194cb9bb17cc9a351457ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } } peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } }
I20260810 18:41:24.695524 7474 raft_consensus.cc:348] T df09f7d1d4064bc89b78728d3f773585 P 6a774567050643dcaa202b8bc5fd2946 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } } peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } }
I20260810 18:41:24.695475 7484 tablet_replica.cc:439] T df09f7d1d4064bc89b78728d3f773585 P 444dd6af1e194cb9bb17cc9a351457ea: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } } peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } } }
I20260810 18:41:24.695993 7474 raft_consensus.cc:374] T df09f7d1d4064bc89b78728d3f773585 P 6a774567050643dcaa202b8bc5fd2946 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:24.696062 7484 tablet_replica.cc:442] T df09f7d1d4064bc89b78728d3f773585 P 444dd6af1e194cb9bb17cc9a351457ea: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:41:24.696180 7474 raft_consensus.cc:729] T df09f7d1d4064bc89b78728d3f773585 P 6a774567050643dcaa202b8bc5fd2946 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a774567050643dcaa202b8bc5fd2946, State: Initialized, Role: FOLLOWER
I20260810 18:41:24.696619 7474 consensus_queue.cc:260] T df09f7d1d4064bc89b78728d3f773585 P 6a774567050643dcaa202b8bc5fd2946 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } } peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } }
I20260810 18:41:24.697160 7475 tablet_bootstrap.cc:492] T df09f7d1d4064bc89b78728d3f773585 P 54b83db5272749afbdc9a6d659eed5d2: No bootstrap required, opened a new log
I20260810 18:41:24.697463 7475 ts_tablet_manager.cc:1397] T df09f7d1d4064bc89b78728d3f773585 P 54b83db5272749afbdc9a6d659eed5d2: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.004s
I20260810 18:41:24.697220 7502 tablet_replica.cc:439] T df09f7d1d4064bc89b78728d3f773585 P 6a774567050643dcaa202b8bc5fd2946: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } } peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } } }
I20260810 18:41:24.697711 7502 tablet_replica.cc:442] T df09f7d1d4064bc89b78728d3f773585 P 6a774567050643dcaa202b8bc5fd2946: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:41:24.698851 7475 raft_consensus.cc:348] T df09f7d1d4064bc89b78728d3f773585 P 54b83db5272749afbdc9a6d659eed5d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } } peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } }
I20260810 18:41:24.699384 7475 raft_consensus.cc:374] T df09f7d1d4064bc89b78728d3f773585 P 54b83db5272749afbdc9a6d659eed5d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:24.699611 7475 raft_consensus.cc:729] T df09f7d1d4064bc89b78728d3f773585 P 54b83db5272749afbdc9a6d659eed5d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54b83db5272749afbdc9a6d659eed5d2, State: Initialized, Role: FOLLOWER
I20260810 18:41:24.699671 7474 ts_tablet_manager.cc:1428] T df09f7d1d4064bc89b78728d3f773585 P 6a774567050643dcaa202b8bc5fd2946: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20260810 18:41:24.700194 7475 consensus_queue.cc:260] T df09f7d1d4064bc89b78728d3f773585 P 54b83db5272749afbdc9a6d659eed5d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } } peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } }
I20260810 18:41:24.701068 7475 ts_tablet_manager.cc:1428] T df09f7d1d4064bc89b78728d3f773585 P 54b83db5272749afbdc9a6d659eed5d2: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 18:41:24.701462 7476 ts_tablet_manager.cc:1428] T df09f7d1d4064bc89b78728d3f773585 P 444dd6af1e194cb9bb17cc9a351457ea: Time spent starting tablet: real 0.008s user 0.000s sys 0.003s
I20260810 18:41:24.702299 7483 tablet_replica.cc:439] T df09f7d1d4064bc89b78728d3f773585 P 54b83db5272749afbdc9a6d659eed5d2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } } peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } } }
I20260810 18:41:24.702824 7483 tablet_replica.cc:442] T df09f7d1d4064bc89b78728d3f773585 P 54b83db5272749afbdc9a6d659eed5d2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:41:24.925475 7502 consensus_queue.cc:1035] T 079466a76e5648b88dfeb29954275e39 P 6a774567050643dcaa202b8bc5fd2946 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:41:24.933228 7542 consensus_queue.cc:1035] T 079466a76e5648b88dfeb29954275e39 P 6a774567050643dcaa202b8bc5fd2946 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 18:41:25.093109 7542 raft_consensus.cc:482] T df09f7d1d4064bc89b78728d3f773585 P 6a774567050643dcaa202b8bc5fd2946 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:41:25.093495 7542 raft_consensus.cc:504] T df09f7d1d4064bc89b78728d3f773585 P 6a774567050643dcaa202b8bc5fd2946 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } } peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } }
I20260810 18:41:25.094439 7542 leader_election.cc:288] T df09f7d1d4064bc89b78728d3f773585 P 6a774567050643dcaa202b8bc5fd2946 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 444dd6af1e194cb9bb17cc9a351457ea (127.4.228.193:45063), 54b83db5272749afbdc9a6d659eed5d2 (127.4.228.194:33349)
I20260810 18:41:25.095191 6942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df09f7d1d4064bc89b78728d3f773585" candidate_uuid: "6a774567050643dcaa202b8bc5fd2946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444dd6af1e194cb9bb17cc9a351457ea" is_pre_election: true
I20260810 18:41:25.095269 7151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df09f7d1d4064bc89b78728d3f773585" candidate_uuid: "6a774567050643dcaa202b8bc5fd2946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54b83db5272749afbdc9a6d659eed5d2" is_pre_election: true
I20260810 18:41:25.095654 7151 raft_consensus.cc:2454] T df09f7d1d4064bc89b78728d3f773585 P 54b83db5272749afbdc9a6d659eed5d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a774567050643dcaa202b8bc5fd2946 in term 0.
I20260810 18:41:25.095590 6942 raft_consensus.cc:2454] T df09f7d1d4064bc89b78728d3f773585 P 444dd6af1e194cb9bb17cc9a351457ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a774567050643dcaa202b8bc5fd2946 in term 0.
I20260810 18:41:25.096487 7267 leader_election.cc:302] T df09f7d1d4064bc89b78728d3f773585 P 6a774567050643dcaa202b8bc5fd2946 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 444dd6af1e194cb9bb17cc9a351457ea, 54b83db5272749afbdc9a6d659eed5d2, 6a774567050643dcaa202b8bc5fd2946; no voters:
I20260810 18:41:25.096995 7542 raft_consensus.cc:2781] T df09f7d1d4064bc89b78728d3f773585 P 6a774567050643dcaa202b8bc5fd2946 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:41:25.097105 7542 raft_consensus.cc:482] T df09f7d1d4064bc89b78728d3f773585 P 6a774567050643dcaa202b8bc5fd2946 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:41:25.097364 7542 raft_consensus.cc:3037] T df09f7d1d4064bc89b78728d3f773585 P 6a774567050643dcaa202b8bc5fd2946 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:25.099689 7542 raft_consensus.cc:504] T df09f7d1d4064bc89b78728d3f773585 P 6a774567050643dcaa202b8bc5fd2946 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } } peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } }
I20260810 18:41:25.100261 7542 leader_election.cc:288] T df09f7d1d4064bc89b78728d3f773585 P 6a774567050643dcaa202b8bc5fd2946 [CANDIDATE]: Term 1 election: Requested vote from peers 444dd6af1e194cb9bb17cc9a351457ea (127.4.228.193:45063), 54b83db5272749afbdc9a6d659eed5d2 (127.4.228.194:33349)
I20260810 18:41:25.100925 6942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df09f7d1d4064bc89b78728d3f773585" candidate_uuid: "6a774567050643dcaa202b8bc5fd2946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "444dd6af1e194cb9bb17cc9a351457ea"
I20260810 18:41:25.100960 7151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df09f7d1d4064bc89b78728d3f773585" candidate_uuid: "6a774567050643dcaa202b8bc5fd2946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54b83db5272749afbdc9a6d659eed5d2"
I20260810 18:41:25.101199 6942 raft_consensus.cc:3037] T df09f7d1d4064bc89b78728d3f773585 P 444dd6af1e194cb9bb17cc9a351457ea [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:25.101362 7151 raft_consensus.cc:3037] T df09f7d1d4064bc89b78728d3f773585 P 54b83db5272749afbdc9a6d659eed5d2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:25.103355 6942 raft_consensus.cc:2454] T df09f7d1d4064bc89b78728d3f773585 P 444dd6af1e194cb9bb17cc9a351457ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a774567050643dcaa202b8bc5fd2946 in term 1.
I20260810 18:41:25.103996 7269 leader_election.cc:302] T df09f7d1d4064bc89b78728d3f773585 P 6a774567050643dcaa202b8bc5fd2946 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 444dd6af1e194cb9bb17cc9a351457ea, 6a774567050643dcaa202b8bc5fd2946; no voters:
I20260810 18:41:25.104313 7151 raft_consensus.cc:2454] T df09f7d1d4064bc89b78728d3f773585 P 54b83db5272749afbdc9a6d659eed5d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a774567050643dcaa202b8bc5fd2946 in term 1.
I20260810 18:41:25.104411 7542 raft_consensus.cc:2781] T df09f7d1d4064bc89b78728d3f773585 P 6a774567050643dcaa202b8bc5fd2946 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:41:25.104743 7542 raft_consensus.cc:686] T df09f7d1d4064bc89b78728d3f773585 P 6a774567050643dcaa202b8bc5fd2946 [term 1 LEADER]: Becoming Leader. State: Replica: 6a774567050643dcaa202b8bc5fd2946, State: Running, Role: LEADER
I20260810 18:41:25.105173 7542 consensus_queue.cc:237] T df09f7d1d4064bc89b78728d3f773585 P 6a774567050643dcaa202b8bc5fd2946 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } } peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } }
I20260810 18:41:25.107244 7502 tablet_replica.cc:439] T df09f7d1d4064bc89b78728d3f773585 P 6a774567050643dcaa202b8bc5fd2946: TxnStatusTablet state changed. Reason: New leader 6a774567050643dcaa202b8bc5fd2946. Latest consensus state: current_term: 1 leader_uuid: "6a774567050643dcaa202b8bc5fd2946" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } } peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } } }
I20260810 18:41:25.107559 7502 tablet_replica.cc:442] T df09f7d1d4064bc89b78728d3f773585 P 6a774567050643dcaa202b8bc5fd2946: This TxnStatusTablet replica's current role is: LEADER
I20260810 18:41:25.108886 7549 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 18:41:25.109289 7549 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 18:41:25.109706 6693 catalog_manager.cc:5495] T df09f7d1d4064bc89b78728d3f773585 P 6a774567050643dcaa202b8bc5fd2946 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a774567050643dcaa202b8bc5fd2946 (127.4.228.195). New cstate: current_term: 1 leader_uuid: "6a774567050643dcaa202b8bc5fd2946" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } health_report { overall_health: HEALTHY } } }
I20260810 18:41:25.164283 7553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:25.148947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:38957 (local address 127.0.0.1:37214)
0810 18:41:25.149722 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:25.149745 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:25.150093 (+ 348us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:25.154959 (+ 4866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:25.154967 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:25.154989 (+ 22us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:41:25.155622 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:25.155640 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:25.157660 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:25.157667 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:25.160386 (+ 2719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:25.160405 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:25.161976 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:25.161985 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:25.162292 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:25.162417 (+ 125us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:41:25.163155 (+ 738us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:41:25.163188 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 18:41:25.163318 (+ 130us) client_negotiation.cc:240] Negotiation successful
0810 18:41:25.163520 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":622,"threads_started":1}
I20260810 18:41:25.164491 7554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:25.149373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37214 (local address 127.4.228.195:38957)
0810 18:41:25.154305 (+ 4932us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:25.154321 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:25.154377 (+ 56us) server_negotiation.cc:407] Connection header received
0810 18:41:25.154617 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:25.154623 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:25.154726 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:25.154851 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:41:25.155949 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:25.157472 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:25.160572 (+ 3100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:25.161716 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:25.161866 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:25.162846 (+ 980us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:41:25.163006 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:41:25.163127 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:25.163606 (+ 479us) server_negotiation.cc:299] Negotiation successful
0810 18:41:25.163729 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4605,"thread_start_us":4254,"threads_started":1}
I20260810 18:41:25.171216 6942 raft_consensus.cc:1261] T df09f7d1d4064bc89b78728d3f773585 P 444dd6af1e194cb9bb17cc9a351457ea [term 1 FOLLOWER]: Refusing update from remote peer 6a774567050643dcaa202b8bc5fd2946: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:41:25.172492 7542 consensus_queue.cc:1035] T df09f7d1d4064bc89b78728d3f773585 P 6a774567050643dcaa202b8bc5fd2946 [LEADER]: Connected to new peer: Peer: permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:41:25.172608 7151 raft_consensus.cc:1261] T df09f7d1d4064bc89b78728d3f773585 P 54b83db5272749afbdc9a6d659eed5d2 [term 1 FOLLOWER]: Refusing update from remote peer 6a774567050643dcaa202b8bc5fd2946: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:41:25.173527 7542 consensus_queue.cc:1035] T df09f7d1d4064bc89b78728d3f773585 P 6a774567050643dcaa202b8bc5fd2946 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:41:25.179188 7483 tablet_replica.cc:439] T df09f7d1d4064bc89b78728d3f773585 P 54b83db5272749afbdc9a6d659eed5d2: TxnStatusTablet state changed. Reason: New leader 6a774567050643dcaa202b8bc5fd2946. Latest consensus state: current_term: 1 leader_uuid: "6a774567050643dcaa202b8bc5fd2946" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } } peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } } }
I20260810 18:41:25.179617 7483 tablet_replica.cc:442] T df09f7d1d4064bc89b78728d3f773585 P 54b83db5272749afbdc9a6d659eed5d2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:41:25.183118 7484 tablet_replica.cc:439] T df09f7d1d4064bc89b78728d3f773585 P 444dd6af1e194cb9bb17cc9a351457ea: TxnStatusTablet state changed. Reason: New leader 6a774567050643dcaa202b8bc5fd2946. Latest consensus state: current_term: 1 leader_uuid: "6a774567050643dcaa202b8bc5fd2946" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } } peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } } }
I20260810 18:41:25.183646 7484 tablet_replica.cc:442] T df09f7d1d4064bc89b78728d3f773585 P 444dd6af1e194cb9bb17cc9a351457ea: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:41:25.185293 7547 mvcc.cc:205] Tried to move back new op lower bound from 7317042320051404800 to 7317042319793283072. Current Snapshot: MvccSnapshot[applied={T|T < 7317042320051404800}]
I20260810 18:41:25.187495 7540 mvcc.cc:205] Tried to move back new op lower bound from 7317042320051404800 to 7317042319793283072. Current Snapshot: MvccSnapshot[applied={T|T < 7317042320051404800}]
I20260810 18:41:25.189004 7483 tablet_replica.cc:439] T df09f7d1d4064bc89b78728d3f773585 P 54b83db5272749afbdc9a6d659eed5d2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6a774567050643dcaa202b8bc5fd2946" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } } peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } } }
I20260810 18:41:25.189527 7483 tablet_replica.cc:442] T df09f7d1d4064bc89b78728d3f773585 P 54b83db5272749afbdc9a6d659eed5d2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:41:25.190227 7484 tablet_replica.cc:439] T df09f7d1d4064bc89b78728d3f773585 P 444dd6af1e194cb9bb17cc9a351457ea: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6a774567050643dcaa202b8bc5fd2946" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } } peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } } }
I20260810 18:41:25.190641 7484 tablet_replica.cc:442] T df09f7d1d4064bc89b78728d3f773585 P 444dd6af1e194cb9bb17cc9a351457ea: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:41:25.195374 7502 tablet_replica.cc:439] T df09f7d1d4064bc89b78728d3f773585 P 6a774567050643dcaa202b8bc5fd2946: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6a774567050643dcaa202b8bc5fd2946" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } } peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } } }
I20260810 18:41:25.195847 7502 tablet_replica.cc:442] T df09f7d1d4064bc89b78728d3f773585 P 6a774567050643dcaa202b8bc5fd2946: This TxnStatusTablet replica's current role is: LEADER
I20260810 18:41:25.195670 7489 tablet_replica.cc:439] T df09f7d1d4064bc89b78728d3f773585 P 6a774567050643dcaa202b8bc5fd2946: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6a774567050643dcaa202b8bc5fd2946" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "444dd6af1e194cb9bb17cc9a351457ea" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 45063 } } peers { permanent_uuid: "54b83db5272749afbdc9a6d659eed5d2" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 33349 } } peers { permanent_uuid: "6a774567050643dcaa202b8bc5fd2946" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38957 } } }
I20260810 18:41:25.196063 7489 tablet_replica.cc:442] T df09f7d1d4064bc89b78728d3f773585 P 6a774567050643dcaa202b8bc5fd2946: This TxnStatusTablet replica's current role is: LEADER
I20260810 18:41:25.228261 7554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:25.218264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:52915 (local address 127.4.228.195:38957)
0810 18:41:25.218576 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:25.218583 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:25.220347 (+ 1764us) server_negotiation.cc:407] Connection header received
0810 18:41:25.220628 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:25.220634 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:25.220744 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:25.220852 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:41:25.221941 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:25.223512 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:25.225689 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:25.226667 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:25.226814 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:25.227318 (+ 504us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:41:25.227461 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:41:25.227557 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:25.227724 (+ 167us) server_negotiation.cc:299] Negotiation successful
0810 18:41:25.227841 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":64}
I20260810 18:41:25.228327 7562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:25.218535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:38957 (local address 127.4.228.195:52915)
0810 18:41:25.220204 (+ 1669us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:25.220236 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:25.220407 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:25.220895 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:25.220899 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:25.220910 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:41:25.221718 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:25.221730 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:25.223743 (+ 2013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:25.223747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:25.225510 (+ 1763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:25.225526 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:25.226819 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:25.226824 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:25.227038 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:25.227101 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:41:25.227567 (+ 466us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:41:25.227606 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 18:41:25.227674 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 18:41:25.227790 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1427,"mutex_wait_us":1,"thread_start_us":1001,"threads_started":1}
I20260810 18:41:25.319813 6693 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:41:25.320232 6693 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:41:25.324565 6693 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5a659e0d5d194cfa93aa8c8d12e9ccf3: Sending DeleteTablet for 3 replicas of tablet 079466a76e5648b88dfeb29954275e39
I20260810 18:41:25.325810 7121 tablet_service.cc:1514] Processing DeleteTablet for tablet 079466a76e5648b88dfeb29954275e39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:41:25 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:51930
I20260810 18:41:25.325834 6909 tablet_service.cc:1514] Processing DeleteTablet for tablet 079466a76e5648b88dfeb29954275e39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:41:25 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:47016
I20260810 18:41:25.326599 7342 tablet_service.cc:1514] Processing DeleteTablet for tablet 079466a76e5648b88dfeb29954275e39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:41:25 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:37198
I20260810 18:41:25.326936 7567 tablet_replica.cc:330] T 079466a76e5648b88dfeb29954275e39 P 54b83db5272749afbdc9a6d659eed5d2: stopping tablet replica
I20260810 18:41:25.327234 7567 raft_consensus.cc:2229] T 079466a76e5648b88dfeb29954275e39 P 54b83db5272749afbdc9a6d659eed5d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:41:25.327653 7567 raft_consensus.cc:2258] T 079466a76e5648b88dfeb29954275e39 P 54b83db5272749afbdc9a6d659eed5d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:41:25.329164 7567 ts_tablet_manager.cc:1905] T 079466a76e5648b88dfeb29954275e39 P 54b83db5272749afbdc9a6d659eed5d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:41:25.341152 7567 ts_tablet_manager.cc:1918] T 079466a76e5648b88dfeb29954275e39 P 54b83db5272749afbdc9a6d659eed5d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:41:25.341415 7567 log.cc:1196] T 079466a76e5648b88dfeb29954275e39 P 54b83db5272749afbdc9a6d659eed5d2: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-1/wal/wals/079466a76e5648b88dfeb29954275e39
I20260810 18:41:25.342187 7567 ts_tablet_manager.cc:1939] T 079466a76e5648b88dfeb29954275e39 P 54b83db5272749afbdc9a6d659eed5d2: Deleting consensus metadata
I20260810 18:41:25.343763 6670 catalog_manager.cc:4841] TS 54b83db5272749afbdc9a6d659eed5d2 (127.4.228.194:33349): tablet 079466a76e5648b88dfeb29954275e39 (table test-table [id=9cf802616fb74d6eb159bb8819b47d34]) successfully deleted
I20260810 18:41:25.349684 7571 tablet_replica.cc:330] T 079466a76e5648b88dfeb29954275e39 P 6a774567050643dcaa202b8bc5fd2946: stopping tablet replica
I20260810 18:41:25.350080 7571 raft_consensus.cc:2229] T 079466a76e5648b88dfeb29954275e39 P 6a774567050643dcaa202b8bc5fd2946 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:41:25.350857 7571 raft_consensus.cc:2258] T 079466a76e5648b88dfeb29954275e39 P 6a774567050643dcaa202b8bc5fd2946 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:41:25.351673 7568 tablet_replica.cc:330] T 079466a76e5648b88dfeb29954275e39 P 444dd6af1e194cb9bb17cc9a351457ea: stopping tablet replica
I20260810 18:41:25.351992 7568 raft_consensus.cc:2229] T 079466a76e5648b88dfeb29954275e39 P 444dd6af1e194cb9bb17cc9a351457ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:41:25.352425 7568 raft_consensus.cc:2258] T 079466a76e5648b88dfeb29954275e39 P 444dd6af1e194cb9bb17cc9a351457ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:41:25.352861 7571 ts_tablet_manager.cc:1905] T 079466a76e5648b88dfeb29954275e39 P 6a774567050643dcaa202b8bc5fd2946: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:41:25.354015 7568 ts_tablet_manager.cc:1905] T 079466a76e5648b88dfeb29954275e39 P 444dd6af1e194cb9bb17cc9a351457ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:41:25.364807 7568 ts_tablet_manager.cc:1918] T 079466a76e5648b88dfeb29954275e39 P 444dd6af1e194cb9bb17cc9a351457ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:41:25.365062 7568 log.cc:1196] T 079466a76e5648b88dfeb29954275e39 P 444dd6af1e194cb9bb17cc9a351457ea: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-0/wal/wals/079466a76e5648b88dfeb29954275e39
I20260810 18:41:25.365366 7571 ts_tablet_manager.cc:1918] T 079466a76e5648b88dfeb29954275e39 P 6a774567050643dcaa202b8bc5fd2946: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:41:25.365718 7571 log.cc:1196] T 079466a76e5648b88dfeb29954275e39 P 6a774567050643dcaa202b8bc5fd2946: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387267874959-5011-0/minicluster-data/ts-2/wal/wals/079466a76e5648b88dfeb29954275e39
I20260810 18:41:25.366031 7568 ts_tablet_manager.cc:1939] T 079466a76e5648b88dfeb29954275e39 P 444dd6af1e194cb9bb17cc9a351457ea: Deleting consensus metadata
Aug 10 18:41:25 dist-test-slave-7kzw krb5kdc[6632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387283, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.228.193@KRBTEST.COM
I20260810 18:41:25.366807 7571 ts_tablet_manager.cc:1939] T 079466a76e5648b88dfeb29954275e39 P 6a774567050643dcaa202b8bc5fd2946: Deleting consensus metadata
I20260810 18:41:25.367990 6671 catalog_manager.cc:4841] TS 444dd6af1e194cb9bb17cc9a351457ea (127.4.228.193:45063): tablet 079466a76e5648b88dfeb29954275e39 (table test-table [id=9cf802616fb74d6eb159bb8819b47d34]) successfully deleted
I20260810 18:41:25.368600 6674 catalog_manager.cc:4841] TS 6a774567050643dcaa202b8bc5fd2946 (127.4.228.195:38957): tablet 079466a76e5648b88dfeb29954275e39 (table test-table [id=9cf802616fb74d6eb159bb8819b47d34]) successfully deleted
I20260810 18:41:25.380086 7573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:25.337762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47032 (local address 127.4.228.193:45063)
0810 18:41:25.338995 (+ 1233us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:25.339003 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:25.347788 (+ 8785us) server_negotiation.cc:407] Connection header received
0810 18:41:25.348138 (+ 350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:25.348145 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:25.348305 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:25.348566 (+ 261us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:25.353285 (+ 4719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:25.354945 (+ 1660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:25.357898 (+ 2953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:25.359156 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:25.359310 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:25.368598 (+ 9288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:25.368622 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:25.368649 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:25.368700 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:25.375075 (+ 6375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:25.376348 (+ 1273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:25.376362 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:25.376367 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:25.376830 (+ 463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:25.377404 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:25.377411 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:25.377416 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:25.378198 (+ 782us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:25.378750 (+ 552us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:25.379027 (+ 277us) server_negotiation.cc:299] Negotiation successful
0810 18:41:25.379354 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":978,"thread_start_us":678,"threads_started":1}
W20260810 18:41:25.381372 7004 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47032
I20260810 18:41:25.405890 7580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:25.385475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57888 (local address 127.4.228.254:45135)
0810 18:41:25.386817 (+ 1342us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:25.386822 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:25.386854 (+ 32us) server_negotiation.cc:407] Connection header received
0810 18:41:25.386910 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:25.386914 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:25.386978 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:25.387106 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:25.389428 (+ 2322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:25.390780 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:25.392431 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:25.393108 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:25.393256 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:25.396977 (+ 3721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:25.396983 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:25.396986 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:25.397025 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:25.401877 (+ 4852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:25.402930 (+ 1053us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:25.402937 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:25.402941 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:25.403181 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:25.403691 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:25.403696 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:25.403698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:25.404138 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:25.404490 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:25.404936 (+ 446us) server_negotiation.cc:299] Negotiation successful
0810 18:41:25.405314 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1159,"thread_start_us":767,"threads_started":1}
W20260810 18:41:25.406965 6693 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57888
I20260810 18:41:25.410032 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 6805
I20260810 18:41:25.436187 7573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:25.411513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47038 (local address 127.4.228.193:45063)
0810 18:41:25.411812 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:25.411823 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:25.419077 (+ 7254us) server_negotiation.cc:407] Connection header received
0810 18:41:25.419325 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:25.419330 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:25.419405 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:25.419524 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:25.420838 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:25.421628 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:25.423516 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:25.424125 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:25.424289 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:25.428116 (+ 3827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:25.428123 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:25.428126 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:25.428169 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:25.432357 (+ 4188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:25.433332 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:25.433337 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:25.433339 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:25.433527 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:25.434087 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:25.434091 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:25.434093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:25.434514 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:25.434958 (+ 444us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:25.435430 (+ 472us) server_negotiation.cc:299] Negotiation successful
0810 18:41:25.435783 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260810 18:41:25.437099 7004 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47038
W20260810 18:41:25.437891 5011 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 6805: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:41:25.438180 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 6805
I20260810 18:41:25.473366 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 7016
Aug 10 18:41:25 dist-test-slave-7kzw krb5kdc[6632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387283, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.228.194@KRBTEST.COM
I20260810 18:41:25.493376 7583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:25.474456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51944 (local address 127.4.228.194:33349)
0810 18:41:25.476174 (+ 1718us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:25.476179 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:25.476198 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:41:25.476259 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:25.476263 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:25.476329 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:25.476465 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:25.478184 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:25.479095 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:25.480563 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:25.481067 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:25.481312 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:25.486256 (+ 4944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:25.486425 (+ 169us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:25.486450 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:25.486582 (+ 132us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:25.489935 (+ 3353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:25.490914 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:25.490924 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:25.490927 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:25.491083 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:25.491539 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:25.491544 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:25.491546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:25.491963 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:25.492322 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:25.492685 (+ 363us) server_negotiation.cc:299] Negotiation successful
0810 18:41:25.492973 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1548,"thread_start_us":1403,"threads_started":1}
W20260810 18:41:25.494344 7219 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51944
W20260810 18:41:25.495252 5011 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 7016: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:41:25.495481 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 7016
I20260810 18:41:25.535184 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 7235
Aug 10 18:41:25 dist-test-slave-7kzw krb5kdc[6632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387283, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.228.195@KRBTEST.COM
I20260810 18:41:25.553869 7554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:25.536182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37218 (local address 127.4.228.195:38957)
0810 18:41:25.536438 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:25.536445 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:25.536469 (+ 24us) server_negotiation.cc:407] Connection header received
0810 18:41:25.536750 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:25.536761 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:25.536840 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:25.536968 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:25.538369 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:25.539400 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:25.540802 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:25.541300 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:25.541451 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:25.546681 (+ 5230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:25.546700 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:25.546721 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:25.546768 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:25.550773 (+ 4005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:25.551574 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:25.551591 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:25.551593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:25.551757 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:25.552152 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:25.552158 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:25.552161 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:25.552569 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:25.552945 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:25.553246 (+ 301us) server_negotiation.cc:299] Negotiation successful
0810 18:41:25.553500 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260810 18:41:25.554653 7437 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37218
W20260810 18:41:25.555496 5011 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 7235: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:41:25.555744 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 7235
I20260810 18:41:25.591297 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 6647
I20260810 18:41:25.592932 6758 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:41:25.715772 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 6647
2026-08-10T18:41:25Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (6037 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:41:25 dist-test-slave-7kzw krb5kdc[7591](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:41:25 dist-test-slave-7kzw krb5kdc[7591](info): set up 2 sockets
Aug 10 18:41:25 dist-test-slave-7kzw krb5kdc[7591](info): commencing operation
krb5kdc: starting...
W20260810 18:41:27.116664 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.343s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:41:27 dist-test-slave-7kzw krb5kdc[7591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387287, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:41:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:41:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.228.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
I20260810 18:41:27.379343 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:44281
--webserver_interface=127.4.228.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:46153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:44281
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:41:27.752592 7610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:41:27.752979 7610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:41:27.753175 7610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:41:27.761780 7610 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:41:27.761924 7610 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 18:41:27.762009 7610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:41:27.762130 7610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:41:27.762195 7610 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:41:27.762328 7610 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:41:27.773263 7610 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:46153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:44281
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:44281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:41:27.774803 7610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:41:27.776286 7610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:41:27.789393 7618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:27.789611 7626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:27.789654 7622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:27.790448 7610 server_base.cc:1034] running on GCE node
I20260810 18:41:27.791391 7610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:41:27.792936 7610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:41:27.794191 7610 hybrid_clock.cc:648] HybridClock initialized: now 1786387287794106 us; error 41 us; skew 500 ppm
I20260810 18:41:27.794662 7610 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:41:27 dist-test-slave-7kzw krb5kdc[7591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387287, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:41:27.798552 7610 init.cc:377] Logged in from keytab as kudu/127.4.228.254@KRBTEST.COM (short username kudu)
I20260810 18:41:27.800418 7610 webserver.cc:460] Webserver started at http://127.4.228.254:39633/ using document root <none> and password file <none>
I20260810 18:41:27.801287 7610 fs_manager.cc:359] Metadata directory not provided
I20260810 18:41:27.801410 7610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:41:27.801801 7610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:41:27.805395 7610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "a3ca18b5d0844726a51e2587276c30d9"
format_stamp: "Formatted at 2026-08-10 18:41:27 on dist-test-slave-7kzw"
server_key: "b1a73892d842c6e861a59a5a522bea64"
server_key_iv: "4e9be4702f9a4f7e4151a769af0a1815"
server_key_version: "encryptionkey@0"
I20260810 18:41:27.806298 7610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "a3ca18b5d0844726a51e2587276c30d9"
format_stamp: "Formatted at 2026-08-10 18:41:27 on dist-test-slave-7kzw"
server_key: "b1a73892d842c6e861a59a5a522bea64"
server_key_iv: "4e9be4702f9a4f7e4151a769af0a1815"
server_key_version: "encryptionkey@0"
I20260810 18:41:27.812685 7610 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260810 18:41:27.817009 7634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:27.818704 7610 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:41:27.818940 7610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "a3ca18b5d0844726a51e2587276c30d9"
format_stamp: "Formatted at 2026-08-10 18:41:27 on dist-test-slave-7kzw"
server_key: "b1a73892d842c6e861a59a5a522bea64"
server_key_iv: "4e9be4702f9a4f7e4151a769af0a1815"
server_key_version: "encryptionkey@0"
I20260810 18:41:27.819193 7610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:41:27.842656 7610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:41:27.847580 7610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:41:27.847929 7610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:41:27.872243 7610 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:44281
I20260810 18:41:27.872263 7721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:44281 every 8 connection(s)
I20260810 18:41:27.873772 7610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:41:27.876606 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 7610
I20260810 18:41:27.876863 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:41:27.877707 5011 external_mini_cluster.cc:1428] Setting key 9b8d12b8f268ecc24b8fb0707801c04e
I20260810 18:41:27.879119 7722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 18:41:27 dist-test-slave-7kzw krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387287, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:27.895531 7722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3ca18b5d0844726a51e2587276c30d9: Bootstrap starting.
I20260810 18:41:27.900424 7722 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3ca18b5d0844726a51e2587276c30d9: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:27.901809 7722 log.cc:824] T 00000000000000000000000000000000 P a3ca18b5d0844726a51e2587276c30d9: Log is configured to *not* fsync() on all Append() calls
I20260810 18:41:27.905242 7722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3ca18b5d0844726a51e2587276c30d9: No bootstrap required, opened a new log
I20260810 18:41:27.905388 7727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:27.880018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35690 (local address 127.4.228.254:44281)
0810 18:41:27.880890 (+ 872us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:27.880902 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:27.880950 (+ 48us) server_negotiation.cc:407] Connection header received
0810 18:41:27.882656 (+ 1706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:27.882683 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:27.883124 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:27.883528 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:27.885742 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:27.887351 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:27.889461 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:27.890240 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:27.890400 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:27.895610 (+ 5210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:27.895624 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:27.895650 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:27.895714 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:27.900274 (+ 4560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:27.901249 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:27.901258 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:27.901268 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:27.901520 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:27.902046 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:27.902054 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:27.902057 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:27.902845 (+ 788us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:27.903222 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:27.903710 (+ 488us) server_negotiation.cc:299] Negotiation successful
0810 18:41:27.904215 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":412,"threads_started":1}
I20260810 18:41:27.914096 7722 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3ca18b5d0844726a51e2587276c30d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3ca18b5d0844726a51e2587276c30d9" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 44281 } }
I20260810 18:41:27.914597 7722 raft_consensus.cc:374] T 00000000000000000000000000000000 P a3ca18b5d0844726a51e2587276c30d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:27.914727 7722 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3ca18b5d0844726a51e2587276c30d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3ca18b5d0844726a51e2587276c30d9, State: Initialized, Role: FOLLOWER
I20260810 18:41:27.915380 7722 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3ca18b5d0844726a51e2587276c30d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3ca18b5d0844726a51e2587276c30d9" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 44281 } }
I20260810 18:41:27.915650 7722 raft_consensus.cc:388] T 00000000000000000000000000000000 P a3ca18b5d0844726a51e2587276c30d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:41:27.915813 7722 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3ca18b5d0844726a51e2587276c30d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:41:27.916008 7722 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3ca18b5d0844726a51e2587276c30d9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:27.919230 7722 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3ca18b5d0844726a51e2587276c30d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3ca18b5d0844726a51e2587276c30d9" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 44281 } }
I20260810 18:41:27.919873 7722 leader_election.cc:302] T 00000000000000000000000000000000 P a3ca18b5d0844726a51e2587276c30d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3ca18b5d0844726a51e2587276c30d9; no voters:
I20260810 18:41:27.920559 7722 leader_election.cc:288] T 00000000000000000000000000000000 P a3ca18b5d0844726a51e2587276c30d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:41:27.920830 7729 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3ca18b5d0844726a51e2587276c30d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:41:27.921615 7729 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3ca18b5d0844726a51e2587276c30d9 [term 1 LEADER]: Becoming Leader. State: Replica: a3ca18b5d0844726a51e2587276c30d9, State: Running, Role: LEADER
I20260810 18:41:27.922302 7729 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3ca18b5d0844726a51e2587276c30d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3ca18b5d0844726a51e2587276c30d9" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 44281 } }
I20260810 18:41:27.923085 7722 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3ca18b5d0844726a51e2587276c30d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:41:27.926357 7732 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3ca18b5d0844726a51e2587276c30d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3ca18b5d0844726a51e2587276c30d9. Latest consensus state: current_term: 1 leader_uuid: "a3ca18b5d0844726a51e2587276c30d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3ca18b5d0844726a51e2587276c30d9" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 44281 } } }
I20260810 18:41:27.927513 7732 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3ca18b5d0844726a51e2587276c30d9 [sys.catalog]: This master's current role is: LEADER
I20260810 18:41:27.927080 7730 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3ca18b5d0844726a51e2587276c30d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a3ca18b5d0844726a51e2587276c30d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3ca18b5d0844726a51e2587276c30d9" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 44281 } } }
I20260810 18:41:27.928144 7730 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3ca18b5d0844726a51e2587276c30d9 [sys.catalog]: This master's current role is: LEADER
I20260810 18:41:27.929277 7741 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:41:27.936285 7741 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:41:27.945773 7741 catalog_manager.cc:1347] Generated new cluster ID: 460c06fe0cd1464b83ecafa9058c7a63
I20260810 18:41:27.945921 7741 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:41:27.986406 7741 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:41:27.988031 7741 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:41:27.999675 7741 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a3ca18b5d0844726a51e2587276c30d9: Generated new TSK 0
I20260810 18:41:28.000808 7741 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.228.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab.
Entry for principal kudu/127.4.228.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab.
I20260810 18:41:28.063386 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.193:0
--local_ip_for_outbound_sockets=127.4.228.193
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:44281
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:46153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:41:28.449651 7758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:41:28.450011 7758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:41:28.450193 7758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:41:28.458832 7758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:41:28.458961 7758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:41:28.459117 7758 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 18:41:28.459203 7758 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 18:41:28.459321 7758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.193
I20260810 18:41:28.469799 7758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:46153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.228.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.228.254:44281
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.193
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:41:28.471386 7758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:41:28.472934 7758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:41:28.487654 7763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:28.487630 7769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:28.487576 7766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:28.488435 7758 server_base.cc:1034] running on GCE node
I20260810 18:41:28.489145 7758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:41:28.490233 7758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:41:28.491475 7758 hybrid_clock.cc:648] HybridClock initialized: now 1786387288491401 us; error 68 us; skew 500 ppm
I20260810 18:41:28.491807 7758 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:41:28 dist-test-slave-7kzw krb5kdc[7591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387288, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:41:28.501015 7758 init.cc:377] Logged in from keytab as kudu/127.4.228.193@KRBTEST.COM (short username kudu)
I20260810 18:41:28.502789 7758 webserver.cc:460] Webserver started at http://127.4.228.193:40877/ using document root <none> and password file <none>
I20260810 18:41:28.503549 7758 fs_manager.cc:359] Metadata directory not provided
I20260810 18:41:28.503662 7758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:41:28.503952 7758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:41:28.507129 7758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-0/data/instance:
uuid: "15095ef6292f44f4a66a82e97497bb20"
format_stamp: "Formatted at 2026-08-10 18:41:28 on dist-test-slave-7kzw"
server_key: "f7d5c934ec540993fcbf9893821321c6"
server_key_iv: "c507a367ae8d6510d31309adab4e6d40"
server_key_version: "encryptionkey@0"
I20260810 18:41:28.507907 7758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance:
uuid: "15095ef6292f44f4a66a82e97497bb20"
format_stamp: "Formatted at 2026-08-10 18:41:28 on dist-test-slave-7kzw"
server_key: "f7d5c934ec540993fcbf9893821321c6"
server_key_iv: "c507a367ae8d6510d31309adab4e6d40"
server_key_version: "encryptionkey@0"
I20260810 18:41:28.513711 7758 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 18:41:28.517814 7779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:28.519737 7758 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 18:41:28.519912 7758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-0/wal
uuid: "15095ef6292f44f4a66a82e97497bb20"
format_stamp: "Formatted at 2026-08-10 18:41:28 on dist-test-slave-7kzw"
server_key: "f7d5c934ec540993fcbf9893821321c6"
server_key_iv: "c507a367ae8d6510d31309adab4e6d40"
server_key_version: "encryptionkey@0"
I20260810 18:41:28.520120 7758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:41:28.545827 7758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:41:28.550990 7758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:41:28.551280 7758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:41:28.554184 7758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:41:28.554363 7758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:28.554525 7758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:41:28.554638 7758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 10 18:41:28 dist-test-slave-7kzw krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387288, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:28.584935 7792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:28.555807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:44281 (local address 127.4.228.193:40933)
0810 18:41:28.556759 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:28.556820 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:28.559077 (+ 2257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:28.560162 (+ 1085us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:28.560179 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:28.561017 (+ 838us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:28.561996 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:28.562022 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:28.563614 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:28.563624 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:28.566905 (+ 3281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:28.566933 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:28.568254 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:28.568266 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:28.568730 (+ 464us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:28.569969 (+ 1239us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:41:28.570012 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:28.574324 (+ 4312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:28.579485 (+ 5161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:28.579505 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:28.579537 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:28.580439 (+ 902us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:28.581078 (+ 639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:28.581083 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:28.581088 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:28.581473 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:28.582343 (+ 870us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:28.582353 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:28.582881 (+ 528us) client_negotiation.cc:769] Sending connection context
0810 18:41:28.583303 (+ 422us) client_negotiation.cc:240] Negotiation successful
0810 18:41:28.583771 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":401,"threads_started":1}
I20260810 18:41:28.586220 7791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:28.555084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:40933 (local address 127.4.228.254:44281)
0810 18:41:28.555682 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:28.555689 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:28.556975 (+ 1286us) server_negotiation.cc:407] Connection header received
0810 18:41:28.559496 (+ 2521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:28.559506 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:28.559623 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:28.559777 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:28.562221 (+ 2444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:28.563408 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:28.567202 (+ 3794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:28.567839 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:28.568034 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:28.574757 (+ 6723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:28.574769 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:28.574776 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:28.574836 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:28.579239 (+ 4403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:28.580692 (+ 1453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:28.580701 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:28.580707 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:28.580893 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:28.581682 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:28.581687 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:28.581690 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:28.582162 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:28.582360 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:28.585706 (+ 3346us) server_negotiation.cc:299] Negotiation successful
0810 18:41:28.585949 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":325,"threads_started":1}
I20260810 18:41:28.604697 7758 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.193:43765
I20260810 18:41:28.604717 7961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.193:43765 every 8 connection(s)
I20260810 18:41:28.606133 7758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
I20260810 18:41:28.608040 7926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:28.592843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:44281 (local address 127.4.228.193:54267)
0810 18:41:28.593486 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:28.593505 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:28.593666 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:28.593991 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:28.593997 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:28.594523 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:28.594923 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:28.594937 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:28.595930 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:28.595940 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:28.597931 (+ 1991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:28.597945 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:28.599198 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:28.599204 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:28.599342 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:28.600218 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:41:28.600249 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:28.601620 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:28.605005 (+ 3385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:28.605011 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:28.605015 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:28.605465 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:28.605907 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:28.605912 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:28.605916 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:28.606051 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:28.606708 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:28.606714 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:28.606948 (+ 234us) client_negotiation.cc:769] Sending connection context
0810 18:41:28.607127 (+ 179us) client_negotiation.cc:240] Negotiation successful
0810 18:41:28.607567 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":360,"threads_started":1}
I20260810 18:41:28.608700 7963 heartbeater.cc:347] Connected to a master server at 127.4.228.254:44281
I20260810 18:41:28.608915 7791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:28.592744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:54267 (local address 127.4.228.254:44281)
0810 18:41:28.592942 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:28.592946 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:28.593595 (+ 649us) server_negotiation.cc:407] Connection header received
0810 18:41:28.593792 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:28.593802 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:28.593864 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:28.593965 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:28.595073 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:28.595784 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:28.598084 (+ 2300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:28.598751 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:28.598922 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:28.601882 (+ 2960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:28.601887 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:28.601890 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:28.601923 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:28.604841 (+ 2918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:28.605592 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:28.605598 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:28.605603 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:28.605787 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:28.606173 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:28.606179 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:28.606183 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:28.606587 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:28.606747 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:28.608397 (+ 1650us) server_negotiation.cc:299] Negotiation successful
0810 18:41:28.608631 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 18:41:28.608991 7963 heartbeater.cc:464] Registering TS with master...
I20260810 18:41:28.609745 7963 heartbeater.cc:511] Master 127.4.228.254:44281 requested a full tablet report, sending...
I20260810 18:41:28.611361 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 7758
I20260810 18:41:28.611596 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance
I20260810 18:41:28.612524 5011 external_mini_cluster.cc:1428] Setting key ddffe31ec67e23b9d695b2b9a8390bec
I20260810 18:41:28.613150 7654 ts_manager.cc:194] Registered new tserver with Master: 15095ef6292f44f4a66a82e97497bb20 (127.4.228.193:43765)
I20260810 18:41:28.615378 7654 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.193@KRBTEST.COM'} at 127.4.228.193:40933
WARNING: no policy specified for kudu/127.4.228.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab.
Entry for principal kudu/127.4.228.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab.
I20260810 18:41:28.670419 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.194:0
--local_ip_for_outbound_sockets=127.4.228.194
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:44281
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:46153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:41:29.041293 7971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:41:29.041662 7971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:41:29.041771 7971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:41:29.049335 7971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:41:29.049479 7971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:41:29.049575 7971 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 18:41:29.049638 7971 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 18:41:29.049710 7971 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.194
I20260810 18:41:29.060063 7971 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:46153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.228.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.228.254:44281
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.194
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:41:29.061480 7971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:41:29.062887 7971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:41:29.077497 7978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:29.077416 7980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:29.077459 7977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:29.078299 7971 server_base.cc:1034] running on GCE node
I20260810 18:41:29.079054 7971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:41:29.080204 7971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:41:29.081470 7971 hybrid_clock.cc:648] HybridClock initialized: now 1786387289081426 us; error 37 us; skew 500 ppm
I20260810 18:41:29.081876 7971 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:41:29 dist-test-slave-7kzw krb5kdc[7591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387289, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:41:29.085570 7971 init.cc:377] Logged in from keytab as kudu/127.4.228.194@KRBTEST.COM (short username kudu)
I20260810 18:41:29.087561 7971 webserver.cc:460] Webserver started at http://127.4.228.194:39979/ using document root <none> and password file <none>
I20260810 18:41:29.088428 7971 fs_manager.cc:359] Metadata directory not provided
I20260810 18:41:29.088593 7971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:41:29.089016 7971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:41:29.092443 7971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-1/data/instance:
uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80"
format_stamp: "Formatted at 2026-08-10 18:41:29 on dist-test-slave-7kzw"
server_key: "7523d429f854fc761c9826d36763af83"
server_key_iv: "bc8b13166c17e77bf67ffcd82ad806f5"
server_key_version: "encryptionkey@0"
I20260810 18:41:29.093330 7971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance:
uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80"
format_stamp: "Formatted at 2026-08-10 18:41:29 on dist-test-slave-7kzw"
server_key: "7523d429f854fc761c9826d36763af83"
server_key_iv: "bc8b13166c17e77bf67ffcd82ad806f5"
server_key_version: "encryptionkey@0"
I20260810 18:41:29.099568 7971 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260810 18:41:29.103681 7990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:29.105260 7971 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:41:29.105429 7971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-1/wal
uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80"
format_stamp: "Formatted at 2026-08-10 18:41:29 on dist-test-slave-7kzw"
server_key: "7523d429f854fc761c9826d36763af83"
server_key_iv: "bc8b13166c17e77bf67ffcd82ad806f5"
server_key_version: "encryptionkey@0"
I20260810 18:41:29.105630 7971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:41:29.120465 7971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:41:29.126068 7971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:41:29.126621 7971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:41:29.130252 7971 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:41:29.130443 7971 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:29.130707 7971 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:41:29.130818 7971 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 18:41:29 dist-test-slave-7kzw krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387289, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:29.156466 7999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:29.130714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:57535 (local address 127.4.228.254:44281)
0810 18:41:29.131331 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:29.131340 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:29.132727 (+ 1387us) server_negotiation.cc:407] Connection header received
0810 18:41:29.135357 (+ 2630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:29.135363 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:29.135469 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:29.135593 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:29.138339 (+ 2746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:29.139527 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:29.142418 (+ 2891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:29.142929 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:29.143078 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:29.148540 (+ 5462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:29.148545 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:29.148549 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:29.148587 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:29.151891 (+ 3304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:29.152906 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:29.152914 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:29.152921 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:29.153103 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:29.153816 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:29.153820 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:29.153822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:29.154205 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:29.154414 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:29.155661 (+ 1247us) server_negotiation.cc:299] Negotiation successful
0810 18:41:29.155934 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":293,"threads_started":1}
I20260810 18:41:29.157662 8003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:29.131628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:44281 (local address 127.4.228.194:57535)
0810 18:41:29.132547 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:29.132593 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:29.134980 (+ 2387us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:29.136081 (+ 1101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:29.136096 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:29.137024 (+ 928us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:29.138010 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:29.138045 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:29.139707 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:29.139717 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:29.142191 (+ 2474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:29.142223 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:29.143217 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:29.143226 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:29.143658 (+ 432us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:29.144586 (+ 928us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:41:29.144620 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:29.148267 (+ 3647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:29.152064 (+ 3797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:29.152075 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:29.152101 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:29.152678 (+ 577us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:29.153267 (+ 589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:29.153271 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:29.153273 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:29.153637 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:29.154583 (+ 946us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:29.154594 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:29.155111 (+ 517us) client_negotiation.cc:769] Sending connection context
0810 18:41:29.155547 (+ 436us) client_negotiation.cc:240] Negotiation successful
0810 18:41:29.156213 (+ 666us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":383,"threads_started":1}
I20260810 18:41:29.179200 7999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:29.164751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:50203 (local address 127.4.228.254:44281)
0810 18:41:29.165013 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:29.165023 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:29.165343 (+ 320us) server_negotiation.cc:407] Connection header received
0810 18:41:29.165501 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:29.165510 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:29.165629 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:29.165780 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:29.166938 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:29.167691 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:29.169079 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:29.169497 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:29.169633 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:29.172005 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:29.172015 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:29.172023 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:29.172085 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:29.176055 (+ 3970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:29.177064 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:29.177069 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:29.177073 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:29.177220 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:29.177597 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:29.177602 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:29.177605 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:29.177963 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:29.178134 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:29.178591 (+ 457us) server_negotiation.cc:299] Negotiation successful
0810 18:41:29.178840 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260810 18:41:29.179370 8084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:29.164020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:44281 (local address 127.4.228.194:50203)
0810 18:41:29.165212 (+ 1192us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:29.165232 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:29.165364 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:29.165803 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:29.165807 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:29.166346 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:29.166777 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:29.166792 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:29.167850 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:29.167855 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:29.168932 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:29.168944 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:29.169707 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:29.169713 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:29.169875 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:29.170568 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:41:29.170586 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:29.171743 (+ 1157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:29.176214 (+ 4471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:29.176219 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:29.176227 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:29.176927 (+ 700us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:29.177337 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:29.177343 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:29.177346 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:29.177485 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:29.178088 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:29.178093 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:29.178293 (+ 200us) client_negotiation.cc:769] Sending connection context
0810 18:41:29.178443 (+ 150us) client_negotiation.cc:240] Negotiation successful
0810 18:41:29.178864 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1031,"thread_start_us":892,"threads_started":1}
I20260810 18:41:29.194937 7971 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.194:43957
I20260810 18:41:29.194963 8157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.194:43957 every 8 connection(s)
I20260810 18:41:29.196401 7971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
I20260810 18:41:29.197274 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 7971
I20260810 18:41:29.197512 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance
I20260810 18:41:29.198194 5011 external_mini_cluster.cc:1428] Setting key 5f09fe03d27ed65c36b20cf94d4985a9
I20260810 18:41:29.198912 8159 heartbeater.cc:347] Connected to a master server at 127.4.228.254:44281
I20260810 18:41:29.199167 8159 heartbeater.cc:464] Registering TS with master...
I20260810 18:41:29.200111 8159 heartbeater.cc:511] Master 127.4.228.254:44281 requested a full tablet report, sending...
I20260810 18:41:29.201629 7654 ts_manager.cc:194] Registered new tserver with Master: 1cc8d7149e8a4a3d898b3043f7a8ba80 (127.4.228.194:43957)
I20260810 18:41:29.202579 7654 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.194@KRBTEST.COM'} at 127.4.228.194:57535
WARNING: no policy specified for kudu/127.4.228.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab.
Entry for principal kudu/127.4.228.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab.
I20260810 18:41:29.255177 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.195:0
--local_ip_for_outbound_sockets=127.4.228.195
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:44281
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:46153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:41:29.618737 7963 heartbeater.cc:503] Master 127.4.228.254:44281 was elected leader, sending a full tablet report...
W20260810 18:41:29.649235 8167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:41:29.649631 8167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:41:29.649789 8167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:41:29.657284 8167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:41:29.657496 8167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:41:29.657662 8167 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 18:41:29.657761 8167 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 18:41:29.657902 8167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.195
I20260810 18:41:29.672091 8167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:46153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.228.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.228.254:44281
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.195
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:41:29.673606 8167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:41:29.675251 8167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:41:29.690099 8175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:29.690171 8177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:29.690135 8179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:29.691192 8167 server_base.cc:1034] running on GCE node
I20260810 18:41:29.691939 8167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:41:29.693066 8167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:41:29.694317 8167 hybrid_clock.cc:648] HybridClock initialized: now 1786387289694258 us; error 50 us; skew 500 ppm
I20260810 18:41:29.694769 8167 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:41:29 dist-test-slave-7kzw krb5kdc[7591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387289, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:41:29.698338 8167 init.cc:377] Logged in from keytab as kudu/127.4.228.195@KRBTEST.COM (short username kudu)
I20260810 18:41:29.700233 8167 webserver.cc:460] Webserver started at http://127.4.228.195:41437/ using document root <none> and password file <none>
I20260810 18:41:29.701048 8167 fs_manager.cc:359] Metadata directory not provided
I20260810 18:41:29.701232 8167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:41:29.701628 8167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:41:29.704810 8167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-2/data/instance:
uuid: "06aed71215d44a608962bc2920f6f5dc"
format_stamp: "Formatted at 2026-08-10 18:41:29 on dist-test-slave-7kzw"
server_key: "ca8d10c8cfaf09a2dc6024c88a606c05"
server_key_iv: "23410f228b74db2249d519d157ecb35e"
server_key_version: "encryptionkey@0"
I20260810 18:41:29.705662 8167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance:
uuid: "06aed71215d44a608962bc2920f6f5dc"
format_stamp: "Formatted at 2026-08-10 18:41:29 on dist-test-slave-7kzw"
server_key: "ca8d10c8cfaf09a2dc6024c88a606c05"
server_key_iv: "23410f228b74db2249d519d157ecb35e"
server_key_version: "encryptionkey@0"
I20260810 18:41:29.711664 8167 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260810 18:41:29.715885 8189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:29.717442 8167 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:41:29.717711 8167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-2/wal
uuid: "06aed71215d44a608962bc2920f6f5dc"
format_stamp: "Formatted at 2026-08-10 18:41:29 on dist-test-slave-7kzw"
server_key: "ca8d10c8cfaf09a2dc6024c88a606c05"
server_key_iv: "23410f228b74db2249d519d157ecb35e"
server_key_version: "encryptionkey@0"
I20260810 18:41:29.717963 8167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:41:29.742141 8167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:41:29.746908 8167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:41:29.747485 8167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:41:29.750958 8167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:41:29.751107 8167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:29.751236 8167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:41:29.751354 8167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 18:41:29 dist-test-slave-7kzw krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387289, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:29.779335 8205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:29.751723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:41539 (local address 127.4.228.254:44281)
0810 18:41:29.752429 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:29.752434 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:29.753650 (+ 1216us) server_negotiation.cc:407] Connection header received
0810 18:41:29.755870 (+ 2220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:29.755876 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:29.755976 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:29.756116 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:29.758856 (+ 2740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:29.760080 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:29.762814 (+ 2734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:29.763656 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:29.763853 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:29.769872 (+ 6019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:29.769885 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:29.769894 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:29.769962 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:29.775175 (+ 5213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:29.776446 (+ 1271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:29.776453 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:29.776460 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:29.776676 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:29.777298 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:29.777306 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:29.777314 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:29.777619 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:29.777789 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:29.778643 (+ 854us) server_negotiation.cc:299] Negotiation successful
0810 18:41:29.778888 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":456,"threads_started":1}
I20260810 18:41:29.781078 8206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:29.752468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:44281 (local address 127.4.228.195:41539)
0810 18:41:29.753399 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:29.753448 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:29.755550 (+ 2102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:29.756444 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:29.756462 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:29.757361 (+ 899us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:29.758645 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:29.758671 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:29.760266 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:29.760302 (+ 36us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:29.762613 (+ 2311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:29.762641 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:29.764094 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:29.764112 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:29.764481 (+ 369us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:29.765520 (+ 1039us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:41:29.765571 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:29.769464 (+ 3893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:29.775419 (+ 5955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:29.775437 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:29.775465 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:29.776206 (+ 741us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:29.776852 (+ 646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:29.776858 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:29.776862 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:29.777142 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:29.777764 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:29.777773 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:29.778193 (+ 420us) client_negotiation.cc:769] Sending connection context
0810 18:41:29.778520 (+ 327us) client_negotiation.cc:240] Negotiation successful
0810 18:41:29.779181 (+ 661us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":660,"thread_start_us":417,"threads_started":1}
I20260810 18:41:29.804965 8205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:29.788676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:53883 (local address 127.4.228.254:44281)
0810 18:41:29.788917 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:29.788927 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:29.789645 (+ 718us) server_negotiation.cc:407] Connection header received
0810 18:41:29.789884 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:29.789888 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:29.789956 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:29.790043 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:29.792050 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:29.793044 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:29.794445 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:29.794981 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:29.795128 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:29.798013 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:29.798021 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:29.798027 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:29.798074 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:29.801513 (+ 3439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:29.802603 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:29.802610 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:29.802614 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:29.802809 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:29.803381 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:29.803385 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:29.803387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:29.803701 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:29.803824 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:29.804494 (+ 670us) server_negotiation.cc:299] Negotiation successful
0810 18:41:29.804683 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 18:41:29.805670 8310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:29.788654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:44281 (local address 127.4.228.195:53883)
0810 18:41:29.789502 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:29.789532 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:29.789740 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:29.790137 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:29.790145 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:29.791193 (+ 1048us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:29.791836 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:29.791860 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:29.793230 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:29.793236 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:29.794280 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:29.794292 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:29.795260 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:29.795273 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:29.795542 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:29.796422 (+ 880us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:41:29.796452 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:29.797699 (+ 1247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:29.801705 (+ 4006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:29.801709 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:29.801712 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:29.802399 (+ 687us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:29.802966 (+ 567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:29.802975 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:29.802983 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:29.803225 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:29.803837 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:29.803846 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:29.804164 (+ 318us) client_negotiation.cc:769] Sending connection context
0810 18:41:29.804403 (+ 239us) client_negotiation.cc:240] Negotiation successful
0810 18:41:29.804909 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":400,"threads_started":1}
I20260810 18:41:29.817128 8167 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.195:45901
I20260810 18:41:29.817171 8372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.195:45901 every 8 connection(s)
I20260810 18:41:29.818665 8167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
I20260810 18:41:29.821242 8373 heartbeater.cc:347] Connected to a master server at 127.4.228.254:44281
I20260810 18:41:29.821542 8373 heartbeater.cc:464] Registering TS with master...
I20260810 18:41:29.822238 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 8167
I20260810 18:41:29.822448 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance
I20260810 18:41:29.822486 8373 heartbeater.cc:511] Master 127.4.228.254:44281 requested a full tablet report, sending...
I20260810 18:41:29.823051 5011 external_mini_cluster.cc:1428] Setting key e0a73ae2e5852388f64a0ee2a04a462f
I20260810 18:41:29.824491 7654 ts_manager.cc:194] Registered new tserver with Master: 06aed71215d44a608962bc2920f6f5dc (127.4.228.195:45901)
I20260810 18:41:29.825738 7654 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.195@KRBTEST.COM'} at 127.4.228.195:41539
I20260810 18:41:29.826988 5011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 18:41:29 dist-test-slave-7kzw krb5kdc[7591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 18:41:29 dist-test-slave-7kzw krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:29.897183 8205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:29.879757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35706 (local address 127.4.228.254:44281)
0810 18:41:29.879981 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:29.879989 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:29.880475 (+ 486us) server_negotiation.cc:407] Connection header received
0810 18:41:29.880708 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:29.880713 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:29.880787 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:29.880891 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:29.882915 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:29.884342 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:29.885804 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:29.886222 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:29.886339 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:29.891036 (+ 4697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:29.891042 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:29.891046 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:29.891093 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:29.894248 (+ 3155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:29.895100 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:29.895105 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:29.895108 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:29.895258 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:29.895627 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:29.895630 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:29.895632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:29.895966 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:29.896185 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:29.896525 (+ 340us) server_negotiation.cc:299] Negotiation successful
0810 18:41:29.896753 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260810 18:41:29.902545 7651 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35706:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:41:29.910041 7651 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:41:29.950896 8401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:29.926060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50780 (local address 127.4.228.194:43957)
0810 18:41:29.940787 (+ 14727us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:29.940802 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:29.941811 (+ 1009us) server_negotiation.cc:407] Connection header received
0810 18:41:29.941937 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:29.941947 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:29.942209 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:29.942451 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:41:29.943880 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:29.945262 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:29.948076 (+ 2814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:29.949481 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:29.949620 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:29.949845 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:29.950531 (+ 686us) server_negotiation.cc:299] Negotiation successful
0810 18:41:29.950685 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14468,"thread_start_us":14298,"threads_started":1}
I20260810 18:41:29.950928 8398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:29.925061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58080 (local address 127.4.228.193:43765)
0810 18:41:29.942013 (+ 16952us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:29.942026 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:29.942068 (+ 42us) server_negotiation.cc:407] Connection header received
0810 18:41:29.942209 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:29.942221 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:29.942552 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:29.943026 (+ 474us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:41:29.943601 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:29.945263 (+ 1662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:29.947857 (+ 2594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:29.948951 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:29.949112 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:29.949358 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:29.950416 (+ 1058us) server_negotiation.cc:299] Negotiation successful
0810 18:41:29.950634 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16719,"thread_start_us":16551,"threads_started":1}
I20260810 18:41:29.951439 8396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:29.924858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:43765 (local address 127.0.0.1:58080)
0810 18:41:29.925865 (+ 1007us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:29.925920 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:29.926198 (+ 278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:29.942869 (+ 16671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:29.942875 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:29.942909 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:41:29.943424 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:29.943440 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:29.945432 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:29.945440 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:29.947690 (+ 2250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:29.947704 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:29.949171 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:29.949178 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:29.949385 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:29.949450 (+ 65us) client_negotiation.cc:769] Sending connection context
0810 18:41:29.949613 (+ 163us) client_negotiation.cc:240] Negotiation successful
0810 18:41:29.949851 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"mutex_wait_us":67,"thread_start_us":988,"threads_started":1}
I20260810 18:41:29.951510 8400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:29.925433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:43957 (local address 127.0.0.1:50780)
0810 18:41:29.941460 (+ 16027us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:29.941491 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:29.941666 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:29.942525 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:29.942534 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:29.942573 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:41:29.943702 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:29.943709 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:29.945400 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:29.945408 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:29.947871 (+ 2463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:29.947893 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:29.949698 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:29.949705 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:29.949896 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:29.949927 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 18:41:29.950047 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 18:41:29.950188 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15835,"mutex_wait_us":176,"thread_start_us":15634,"threads_started":1}
I20260810 18:41:29.952450 8399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:29.925458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:45901 (local address 127.0.0.1:55444)
0810 18:41:29.943207 (+ 17749us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:29.943235 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:29.943382 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:29.944406 (+ 1024us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:29.944411 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:29.944437 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:41:29.944846 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:29.944858 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:29.947001 (+ 2143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:29.947007 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:29.949297 (+ 2290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:29.949313 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:29.951594 (+ 2281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:29.951603 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:29.951826 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:29.951870 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 18:41:29.951955 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 18:41:29.952103 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17580,"mutex_wait_us":65,"thread_start_us":17409,"threads_started":1}
I20260810 18:41:29.953281 8402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:29.926550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55444 (local address 127.4.228.195:45901)
0810 18:41:29.933537 (+ 6987us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:29.933547 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:29.943373 (+ 9826us) server_negotiation.cc:407] Connection header received
0810 18:41:29.943589 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:29.943597 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:29.943896 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:29.944087 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:41:29.945087 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:29.946852 (+ 1765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:29.949545 (+ 2693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:29.951041 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:29.951183 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:29.951420 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:29.952840 (+ 1420us) server_negotiation.cc:299] Negotiation successful
0810 18:41:29.953034 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6762,"thread_start_us":6609,"threads_started":1}
I20260810 18:41:29.955987 7855 tablet_service.cc:1467] Processing CreateTablet for tablet e1fe31c8a9624a62a21540125c0a5a12 (DEFAULT_TABLE table=test-table [id=67eb84adb3854c1089f583c6b4f1f435]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:41:29.955987 8059 tablet_service.cc:1467] Processing CreateTablet for tablet e1fe31c8a9624a62a21540125c0a5a12 (DEFAULT_TABLE table=test-table [id=67eb84adb3854c1089f583c6b4f1f435]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:41:29.956254 8264 tablet_service.cc:1467] Processing CreateTablet for tablet e1fe31c8a9624a62a21540125c0a5a12 (DEFAULT_TABLE table=test-table [id=67eb84adb3854c1089f583c6b4f1f435]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:41:29.957520 8264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1fe31c8a9624a62a21540125c0a5a12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:29.957561 7855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1fe31c8a9624a62a21540125c0a5a12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:29.957581 8059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1fe31c8a9624a62a21540125c0a5a12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:29.970250 8403 tablet_bootstrap.cc:492] T e1fe31c8a9624a62a21540125c0a5a12 P 06aed71215d44a608962bc2920f6f5dc: Bootstrap starting.
I20260810 18:41:29.973179 8404 tablet_bootstrap.cc:492] T e1fe31c8a9624a62a21540125c0a5a12 P 15095ef6292f44f4a66a82e97497bb20: Bootstrap starting.
I20260810 18:41:29.974082 8405 tablet_bootstrap.cc:492] T e1fe31c8a9624a62a21540125c0a5a12 P 1cc8d7149e8a4a3d898b3043f7a8ba80: Bootstrap starting.
I20260810 18:41:29.974337 8403 tablet_bootstrap.cc:654] T e1fe31c8a9624a62a21540125c0a5a12 P 06aed71215d44a608962bc2920f6f5dc: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:29.975929 8403 log.cc:824] T e1fe31c8a9624a62a21540125c0a5a12 P 06aed71215d44a608962bc2920f6f5dc: Log is configured to *not* fsync() on all Append() calls
I20260810 18:41:29.977607 8405 tablet_bootstrap.cc:654] T e1fe31c8a9624a62a21540125c0a5a12 P 1cc8d7149e8a4a3d898b3043f7a8ba80: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:29.977944 8404 tablet_bootstrap.cc:654] T e1fe31c8a9624a62a21540125c0a5a12 P 15095ef6292f44f4a66a82e97497bb20: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:29.979213 8405 log.cc:824] T e1fe31c8a9624a62a21540125c0a5a12 P 1cc8d7149e8a4a3d898b3043f7a8ba80: Log is configured to *not* fsync() on all Append() calls
I20260810 18:41:29.979444 8404 log.cc:824] T e1fe31c8a9624a62a21540125c0a5a12 P 15095ef6292f44f4a66a82e97497bb20: Log is configured to *not* fsync() on all Append() calls
I20260810 18:41:29.983078 8403 tablet_bootstrap.cc:492] T e1fe31c8a9624a62a21540125c0a5a12 P 06aed71215d44a608962bc2920f6f5dc: No bootstrap required, opened a new log
I20260810 18:41:29.983561 8403 ts_tablet_manager.cc:1397] T e1fe31c8a9624a62a21540125c0a5a12 P 06aed71215d44a608962bc2920f6f5dc: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.004s
I20260810 18:41:29.989675 8404 tablet_bootstrap.cc:492] T e1fe31c8a9624a62a21540125c0a5a12 P 15095ef6292f44f4a66a82e97497bb20: No bootstrap required, opened a new log
I20260810 18:41:29.989708 8405 tablet_bootstrap.cc:492] T e1fe31c8a9624a62a21540125c0a5a12 P 1cc8d7149e8a4a3d898b3043f7a8ba80: No bootstrap required, opened a new log
I20260810 18:41:29.990042 8405 ts_tablet_manager.cc:1397] T e1fe31c8a9624a62a21540125c0a5a12 P 1cc8d7149e8a4a3d898b3043f7a8ba80: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.008s
I20260810 18:41:29.990052 8404 ts_tablet_manager.cc:1397] T e1fe31c8a9624a62a21540125c0a5a12 P 15095ef6292f44f4a66a82e97497bb20: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.003s
I20260810 18:41:29.997022 8403 raft_consensus.cc:348] T e1fe31c8a9624a62a21540125c0a5a12 P 06aed71215d44a608962bc2920f6f5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:29.997625 8403 raft_consensus.cc:374] T e1fe31c8a9624a62a21540125c0a5a12 P 06aed71215d44a608962bc2920f6f5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:29.997830 8403 raft_consensus.cc:729] T e1fe31c8a9624a62a21540125c0a5a12 P 06aed71215d44a608962bc2920f6f5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06aed71215d44a608962bc2920f6f5dc, State: Initialized, Role: FOLLOWER
I20260810 18:41:29.998718 8403 consensus_queue.cc:260] T e1fe31c8a9624a62a21540125c0a5a12 P 06aed71215d44a608962bc2920f6f5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:29.999348 8404 raft_consensus.cc:348] T e1fe31c8a9624a62a21540125c0a5a12 P 15095ef6292f44f4a66a82e97497bb20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:30.000182 8373 heartbeater.cc:503] Master 127.4.228.254:44281 was elected leader, sending a full tablet report...
I20260810 18:41:30.001089 8403 ts_tablet_manager.cc:1428] T e1fe31c8a9624a62a21540125c0a5a12 P 06aed71215d44a608962bc2920f6f5dc: Time spent starting tablet: real 0.017s user 0.008s sys 0.008s
I20260810 18:41:30.001408 8404 raft_consensus.cc:374] T e1fe31c8a9624a62a21540125c0a5a12 P 15095ef6292f44f4a66a82e97497bb20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:30.001555 8404 raft_consensus.cc:729] T e1fe31c8a9624a62a21540125c0a5a12 P 15095ef6292f44f4a66a82e97497bb20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15095ef6292f44f4a66a82e97497bb20, State: Initialized, Role: FOLLOWER
I20260810 18:41:30.002381 8405 raft_consensus.cc:348] T e1fe31c8a9624a62a21540125c0a5a12 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:30.002355 8404 consensus_queue.cc:260] T e1fe31c8a9624a62a21540125c0a5a12 P 15095ef6292f44f4a66a82e97497bb20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:30.002934 8405 raft_consensus.cc:374] T e1fe31c8a9624a62a21540125c0a5a12 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:30.003075 8405 raft_consensus.cc:729] T e1fe31c8a9624a62a21540125c0a5a12 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc8d7149e8a4a3d898b3043f7a8ba80, State: Initialized, Role: FOLLOWER
I20260810 18:41:30.003854 8405 consensus_queue.cc:260] T e1fe31c8a9624a62a21540125c0a5a12 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:30.004995 8404 ts_tablet_manager.cc:1428] T e1fe31c8a9624a62a21540125c0a5a12 P 15095ef6292f44f4a66a82e97497bb20: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260810 18:41:30.006551 8159 heartbeater.cc:503] Master 127.4.228.254:44281 was elected leader, sending a full tablet report...
I20260810 18:41:30.007450 8405 ts_tablet_manager.cc:1428] T e1fe31c8a9624a62a21540125c0a5a12 P 1cc8d7149e8a4a3d898b3043f7a8ba80: Time spent starting tablet: real 0.017s user 0.010s sys 0.009s
I20260810 18:41:30.030566 8416 raft_consensus.cc:482] T e1fe31c8a9624a62a21540125c0a5a12 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:41:30.031010 8416 raft_consensus.cc:504] T e1fe31c8a9624a62a21540125c0a5a12 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:30.032778 8416 leader_election.cc:288] T e1fe31c8a9624a62a21540125c0a5a12 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15095ef6292f44f4a66a82e97497bb20 (127.4.228.193:43765), 06aed71215d44a608962bc2920f6f5dc (127.4.228.195:45901)
I20260810 18:41:30.043824 8398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:30.033336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:56625 (local address 127.4.228.193:43765)
0810 18:41:30.033554 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:30.033561 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:30.035026 (+ 1465us) server_negotiation.cc:407] Connection header received
0810 18:41:30.035145 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:30.035183 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:30.035288 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:30.035398 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:41:30.036254 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.037556 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.040862 (+ 3306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.042347 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.042593 (+ 246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:30.042684 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:30.043370 (+ 686us) server_negotiation.cc:299] Negotiation successful
0810 18:41:30.043535 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 18:41:30.043856 8402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:30.033482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:57481 (local address 127.4.228.195:45901)
0810 18:41:30.033859 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:30.033870 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:30.034694 (+ 824us) server_negotiation.cc:407] Connection header received
0810 18:41:30.035037 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:30.035043 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:30.035143 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:30.035269 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:41:30.036254 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.037556 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.040862 (+ 3306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.042344 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.042603 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:30.042684 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:30.043370 (+ 686us) server_negotiation.cc:299] Negotiation successful
0810 18:41:30.043584 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 18:41:30.044035 8417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:30.033079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:45901 (local address 127.4.228.194:57481)
0810 18:41:30.034531 (+ 1452us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:30.034558 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:30.034784 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:30.035382 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:30.035390 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:30.035426 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:41:30.035960 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:30.035978 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.037752 (+ 1774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.037758 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:30.040602 (+ 2844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:30.040621 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.042803 (+ 2182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.042811 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:30.043088 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:30.043161 (+ 73us) client_negotiation.cc:769] Sending connection context
0810 18:41:30.043281 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 18:41:30.043486 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1236,"mutex_wait_us":287,"thread_start_us":925,"threads_started":1}
I20260810 18:41:30.044078 8418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:30.033245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:43765 (local address 127.4.228.194:56625)
0810 18:41:30.034872 (+ 1627us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:30.034896 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:30.035008 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:30.035438 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:30.035441 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:30.035461 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:41:30.036046 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:30.036059 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.037752 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.037762 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:30.040602 (+ 2840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:30.040621 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.042803 (+ 2182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.042811 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:30.043145 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:30.043174 (+ 29us) client_negotiation.cc:769] Sending connection context
0810 18:41:30.043281 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 18:41:30.043486 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1490,"mutex_wait_us":272,"thread_start_us":1256,"threads_started":1}
I20260810 18:41:30.045403 7886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1fe31c8a9624a62a21540125c0a5a12" candidate_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15095ef6292f44f4a66a82e97497bb20" is_pre_election: true
I20260810 18:41:30.045387 8294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1fe31c8a9624a62a21540125c0a5a12" candidate_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06aed71215d44a608962bc2920f6f5dc" is_pre_election: true
I20260810 18:41:30.045971 7886 raft_consensus.cc:2454] T e1fe31c8a9624a62a21540125c0a5a12 P 15095ef6292f44f4a66a82e97497bb20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cc8d7149e8a4a3d898b3043f7a8ba80 in term 0.
I20260810 18:41:30.045987 8294 raft_consensus.cc:2454] T e1fe31c8a9624a62a21540125c0a5a12 P 06aed71215d44a608962bc2920f6f5dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cc8d7149e8a4a3d898b3043f7a8ba80 in term 0.
I20260810 18:41:30.046775 7991 leader_election.cc:302] T e1fe31c8a9624a62a21540125c0a5a12 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15095ef6292f44f4a66a82e97497bb20, 1cc8d7149e8a4a3d898b3043f7a8ba80; no voters:
I20260810 18:41:30.047253 8416 raft_consensus.cc:2781] T e1fe31c8a9624a62a21540125c0a5a12 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:41:30.047452 8416 raft_consensus.cc:482] T e1fe31c8a9624a62a21540125c0a5a12 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:41:30.047566 8416 raft_consensus.cc:3037] T e1fe31c8a9624a62a21540125c0a5a12 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:30.049875 8416 raft_consensus.cc:504] T e1fe31c8a9624a62a21540125c0a5a12 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:30.050609 8416 leader_election.cc:288] T e1fe31c8a9624a62a21540125c0a5a12 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [CANDIDATE]: Term 1 election: Requested vote from peers 15095ef6292f44f4a66a82e97497bb20 (127.4.228.193:43765), 06aed71215d44a608962bc2920f6f5dc (127.4.228.195:45901)
I20260810 18:41:30.051194 7886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1fe31c8a9624a62a21540125c0a5a12" candidate_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15095ef6292f44f4a66a82e97497bb20"
I20260810 18:41:30.051298 8294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1fe31c8a9624a62a21540125c0a5a12" candidate_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06aed71215d44a608962bc2920f6f5dc"
I20260810 18:41:30.051440 7886 raft_consensus.cc:3037] T e1fe31c8a9624a62a21540125c0a5a12 P 15095ef6292f44f4a66a82e97497bb20 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:30.051640 8294 raft_consensus.cc:3037] T e1fe31c8a9624a62a21540125c0a5a12 P 06aed71215d44a608962bc2920f6f5dc [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:30.054039 8294 raft_consensus.cc:2454] T e1fe31c8a9624a62a21540125c0a5a12 P 06aed71215d44a608962bc2920f6f5dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cc8d7149e8a4a3d898b3043f7a8ba80 in term 1.
I20260810 18:41:30.054235 7886 raft_consensus.cc:2454] T e1fe31c8a9624a62a21540125c0a5a12 P 15095ef6292f44f4a66a82e97497bb20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cc8d7149e8a4a3d898b3043f7a8ba80 in term 1.
I20260810 18:41:30.054752 7994 leader_election.cc:302] T e1fe31c8a9624a62a21540125c0a5a12 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06aed71215d44a608962bc2920f6f5dc, 1cc8d7149e8a4a3d898b3043f7a8ba80; no voters:
I20260810 18:41:30.055137 8416 raft_consensus.cc:2781] T e1fe31c8a9624a62a21540125c0a5a12 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:41:30.055724 8416 raft_consensus.cc:686] T e1fe31c8a9624a62a21540125c0a5a12 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 1 LEADER]: Becoming Leader. State: Replica: 1cc8d7149e8a4a3d898b3043f7a8ba80, State: Running, Role: LEADER
I20260810 18:41:30.056281 8416 consensus_queue.cc:237] T e1fe31c8a9624a62a21540125c0a5a12 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:30.061625 7651 catalog_manager.cc:5495] T e1fe31c8a9624a62a21540125c0a5a12 P 1cc8d7149e8a4a3d898b3043f7a8ba80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1cc8d7149e8a4a3d898b3043f7a8ba80 (127.4.228.194). New cstate: current_term: 1 leader_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } health_report { overall_health: UNKNOWN } } }
W20260810 18:41:30.070673 8375 tablet.cc:2369] T e1fe31c8a9624a62a21540125c0a5a12 P 06aed71215d44a608962bc2920f6f5dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:41:30.109236 7966 tablet.cc:2369] T e1fe31c8a9624a62a21540125c0a5a12 P 15095ef6292f44f4a66a82e97497bb20: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 18:41:30 dist-test-slave-7kzw krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387287, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:30.126912 8429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:30.109553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:44281 (local address 127.0.0.1:35712)
0810 18:41:30.110392 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:30.110410 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:30.110667 (+ 257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:30.111148 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:30.111152 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:30.112210 (+ 1058us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:30.112572 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:30.112588 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.113737 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.113742 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:30.114973 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:30.114989 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.115868 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.115874 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:30.116016 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:30.116862 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:41:30.116884 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:30.119501 (+ 2617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:30.123352 (+ 3851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:30.123367 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:30.123375 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:30.124112 (+ 737us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:30.124689 (+ 577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:30.124696 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:30.124701 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:30.124948 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:30.125887 (+ 939us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:30.125894 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:30.126085 (+ 191us) client_negotiation.cc:769] Sending connection context
0810 18:41:30.126239 (+ 154us) client_negotiation.cc:240] Negotiation successful
0810 18:41:30.126546 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":470,"threads_started":1}
I20260810 18:41:30.127219 8205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:30.109512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35712 (local address 127.4.228.254:44281)
0810 18:41:30.109761 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:30.109769 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:30.110584 (+ 815us) server_negotiation.cc:407] Connection header received
0810 18:41:30.110854 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:30.110859 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:30.110934 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:30.111046 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:30.112767 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.113568 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.115142 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.115600 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.115763 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:30.119758 (+ 3995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:30.119764 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:30.119768 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:30.119805 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:30.123122 (+ 3317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:30.124303 (+ 1181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:30.124307 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:30.124310 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:30.124514 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:30.125164 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:30.125168 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:30.125171 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:30.125707 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:30.125868 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:30.126373 (+ 505us) server_negotiation.cc:299] Negotiation successful
0810 18:41:30.126788 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":155,"server-negotiator.queue_time_us":95}
I20260810 18:41:30.133361 7651 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:35712:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 18:41:30.134656 7651 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:41:30.143198 8264 tablet_service.cc:1467] Processing CreateTablet for tablet 6361527cfd3640f39308976247403a55 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f28ba50af5ac4520b20a6688a54bf352]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 18:41:30.143021 8059 tablet_service.cc:1467] Processing CreateTablet for tablet 6361527cfd3640f39308976247403a55 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f28ba50af5ac4520b20a6688a54bf352]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 18:41:30.143021 7855 tablet_service.cc:1467] Processing CreateTablet for tablet 6361527cfd3640f39308976247403a55 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f28ba50af5ac4520b20a6688a54bf352]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 18:41:30.144131 8264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6361527cfd3640f39308976247403a55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:30.144165 7855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6361527cfd3640f39308976247403a55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:30.144176 8059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6361527cfd3640f39308976247403a55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:30.150599 8403 tablet_bootstrap.cc:492] T 6361527cfd3640f39308976247403a55 P 06aed71215d44a608962bc2920f6f5dc: Bootstrap starting.
I20260810 18:41:30.152166 8403 tablet_bootstrap.cc:654] T 6361527cfd3640f39308976247403a55 P 06aed71215d44a608962bc2920f6f5dc: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:30.152595 8404 tablet_bootstrap.cc:492] T 6361527cfd3640f39308976247403a55 P 15095ef6292f44f4a66a82e97497bb20: Bootstrap starting.
I20260810 18:41:30.152618 8405 tablet_bootstrap.cc:492] T 6361527cfd3640f39308976247403a55 P 1cc8d7149e8a4a3d898b3043f7a8ba80: Bootstrap starting.
I20260810 18:41:30.154610 8404 tablet_bootstrap.cc:654] T 6361527cfd3640f39308976247403a55 P 15095ef6292f44f4a66a82e97497bb20: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:30.154733 8405 tablet_bootstrap.cc:654] T 6361527cfd3640f39308976247403a55 P 1cc8d7149e8a4a3d898b3043f7a8ba80: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:30.155632 8403 tablet_bootstrap.cc:492] T 6361527cfd3640f39308976247403a55 P 06aed71215d44a608962bc2920f6f5dc: No bootstrap required, opened a new log
I20260810 18:41:30.155929 8403 ts_tablet_manager.cc:1397] T 6361527cfd3640f39308976247403a55 P 06aed71215d44a608962bc2920f6f5dc: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260810 18:41:30.156838 8403 raft_consensus.cc:348] T 6361527cfd3640f39308976247403a55 P 06aed71215d44a608962bc2920f6f5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:30.157121 8403 raft_consensus.cc:374] T 6361527cfd3640f39308976247403a55 P 06aed71215d44a608962bc2920f6f5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:30.157266 8403 raft_consensus.cc:729] T 6361527cfd3640f39308976247403a55 P 06aed71215d44a608962bc2920f6f5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06aed71215d44a608962bc2920f6f5dc, State: Initialized, Role: FOLLOWER
I20260810 18:41:30.157354 8404 tablet_bootstrap.cc:492] T 6361527cfd3640f39308976247403a55 P 15095ef6292f44f4a66a82e97497bb20: No bootstrap required, opened a new log
I20260810 18:41:30.157500 8404 ts_tablet_manager.cc:1397] T 6361527cfd3640f39308976247403a55 P 15095ef6292f44f4a66a82e97497bb20: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260810 18:41:30.157613 8405 tablet_bootstrap.cc:492] T 6361527cfd3640f39308976247403a55 P 1cc8d7149e8a4a3d898b3043f7a8ba80: No bootstrap required, opened a new log
I20260810 18:41:30.157625 8403 consensus_queue.cc:260] T 6361527cfd3640f39308976247403a55 P 06aed71215d44a608962bc2920f6f5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:30.157786 8405 ts_tablet_manager.cc:1397] T 6361527cfd3640f39308976247403a55 P 1cc8d7149e8a4a3d898b3043f7a8ba80: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 18:41:30.158140 8412 tablet_replica.cc:439] T 6361527cfd3640f39308976247403a55 P 06aed71215d44a608962bc2920f6f5dc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } } }
I20260810 18:41:30.158345 8404 raft_consensus.cc:348] T 6361527cfd3640f39308976247403a55 P 15095ef6292f44f4a66a82e97497bb20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:30.158566 8412 tablet_replica.cc:442] T 6361527cfd3640f39308976247403a55 P 06aed71215d44a608962bc2920f6f5dc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:41:30.158638 8404 raft_consensus.cc:374] T 6361527cfd3640f39308976247403a55 P 15095ef6292f44f4a66a82e97497bb20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:30.158739 8404 raft_consensus.cc:729] T 6361527cfd3640f39308976247403a55 P 15095ef6292f44f4a66a82e97497bb20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15095ef6292f44f4a66a82e97497bb20, State: Initialized, Role: FOLLOWER
I20260810 18:41:30.158702 8405 raft_consensus.cc:348] T 6361527cfd3640f39308976247403a55 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:30.158977 8405 raft_consensus.cc:374] T 6361527cfd3640f39308976247403a55 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:30.158967 8404 consensus_queue.cc:260] T 6361527cfd3640f39308976247403a55 P 15095ef6292f44f4a66a82e97497bb20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:30.159088 8405 raft_consensus.cc:729] T 6361527cfd3640f39308976247403a55 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc8d7149e8a4a3d898b3043f7a8ba80, State: Initialized, Role: FOLLOWER
I20260810 18:41:30.159348 8405 consensus_queue.cc:260] T 6361527cfd3640f39308976247403a55 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:30.159626 8404 ts_tablet_manager.cc:1428] T 6361527cfd3640f39308976247403a55 P 15095ef6292f44f4a66a82e97497bb20: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 18:41:30.159464 8415 tablet_replica.cc:439] T 6361527cfd3640f39308976247403a55 P 15095ef6292f44f4a66a82e97497bb20: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } } }
I20260810 18:41:30.159888 8415 tablet_replica.cc:442] T 6361527cfd3640f39308976247403a55 P 15095ef6292f44f4a66a82e97497bb20: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:41:30.160002 8405 ts_tablet_manager.cc:1428] T 6361527cfd3640f39308976247403a55 P 1cc8d7149e8a4a3d898b3043f7a8ba80: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 18:41:30.160974 8416 tablet_replica.cc:439] T 6361527cfd3640f39308976247403a55 P 1cc8d7149e8a4a3d898b3043f7a8ba80: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } } }
I20260810 18:41:30.161278 8416 tablet_replica.cc:442] T 6361527cfd3640f39308976247403a55 P 1cc8d7149e8a4a3d898b3043f7a8ba80: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:41:30.161895 8403 ts_tablet_manager.cc:1428] T 6361527cfd3640f39308976247403a55 P 06aed71215d44a608962bc2920f6f5dc: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
W20260810 18:41:30.199005 8161 tablet.cc:2369] T 6361527cfd3640f39308976247403a55 P 1cc8d7149e8a4a3d898b3043f7a8ba80: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:41:30.324497 8416 raft_consensus.cc:482] T 6361527cfd3640f39308976247403a55 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:41:30.324800 8416 raft_consensus.cc:504] T 6361527cfd3640f39308976247403a55 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:30.325754 8416 leader_election.cc:288] T 6361527cfd3640f39308976247403a55 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15095ef6292f44f4a66a82e97497bb20 (127.4.228.193:43765), 06aed71215d44a608962bc2920f6f5dc (127.4.228.195:45901)
I20260810 18:41:30.326376 8294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6361527cfd3640f39308976247403a55" candidate_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06aed71215d44a608962bc2920f6f5dc" is_pre_election: true
I20260810 18:41:30.326427 7886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6361527cfd3640f39308976247403a55" candidate_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15095ef6292f44f4a66a82e97497bb20" is_pre_election: true
I20260810 18:41:30.326714 8294 raft_consensus.cc:2454] T 6361527cfd3640f39308976247403a55 P 06aed71215d44a608962bc2920f6f5dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cc8d7149e8a4a3d898b3043f7a8ba80 in term 0.
I20260810 18:41:30.326782 7886 raft_consensus.cc:2454] T 6361527cfd3640f39308976247403a55 P 15095ef6292f44f4a66a82e97497bb20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cc8d7149e8a4a3d898b3043f7a8ba80 in term 0.
I20260810 18:41:30.327206 7994 leader_election.cc:302] T 6361527cfd3640f39308976247403a55 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06aed71215d44a608962bc2920f6f5dc, 1cc8d7149e8a4a3d898b3043f7a8ba80; no voters:
I20260810 18:41:30.327546 8416 raft_consensus.cc:2781] T 6361527cfd3640f39308976247403a55 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:41:30.327648 8416 raft_consensus.cc:482] T 6361527cfd3640f39308976247403a55 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:41:30.327806 8416 raft_consensus.cc:3037] T 6361527cfd3640f39308976247403a55 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:30.330299 8416 raft_consensus.cc:504] T 6361527cfd3640f39308976247403a55 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:30.331085 8416 leader_election.cc:288] T 6361527cfd3640f39308976247403a55 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [CANDIDATE]: Term 1 election: Requested vote from peers 15095ef6292f44f4a66a82e97497bb20 (127.4.228.193:43765), 06aed71215d44a608962bc2920f6f5dc (127.4.228.195:45901)
I20260810 18:41:30.331563 7886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6361527cfd3640f39308976247403a55" candidate_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15095ef6292f44f4a66a82e97497bb20"
I20260810 18:41:30.331817 7886 raft_consensus.cc:3037] T 6361527cfd3640f39308976247403a55 P 15095ef6292f44f4a66a82e97497bb20 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:30.331715 8294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6361527cfd3640f39308976247403a55" candidate_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06aed71215d44a608962bc2920f6f5dc"
I20260810 18:41:30.332031 8294 raft_consensus.cc:3037] T 6361527cfd3640f39308976247403a55 P 06aed71215d44a608962bc2920f6f5dc [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:30.334125 7886 raft_consensus.cc:2454] T 6361527cfd3640f39308976247403a55 P 15095ef6292f44f4a66a82e97497bb20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cc8d7149e8a4a3d898b3043f7a8ba80 in term 1.
I20260810 18:41:30.334184 8294 raft_consensus.cc:2454] T 6361527cfd3640f39308976247403a55 P 06aed71215d44a608962bc2920f6f5dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cc8d7149e8a4a3d898b3043f7a8ba80 in term 1.
I20260810 18:41:30.334661 7991 leader_election.cc:302] T 6361527cfd3640f39308976247403a55 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15095ef6292f44f4a66a82e97497bb20, 1cc8d7149e8a4a3d898b3043f7a8ba80; no voters:
I20260810 18:41:30.335038 8416 raft_consensus.cc:2781] T 6361527cfd3640f39308976247403a55 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:41:30.335291 8416 raft_consensus.cc:686] T 6361527cfd3640f39308976247403a55 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 1 LEADER]: Becoming Leader. State: Replica: 1cc8d7149e8a4a3d898b3043f7a8ba80, State: Running, Role: LEADER
I20260810 18:41:30.335584 8416 consensus_queue.cc:237] T 6361527cfd3640f39308976247403a55 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:30.337414 8420 tablet_replica.cc:439] T 6361527cfd3640f39308976247403a55 P 1cc8d7149e8a4a3d898b3043f7a8ba80: TxnStatusTablet state changed. Reason: New leader 1cc8d7149e8a4a3d898b3043f7a8ba80. Latest consensus state: current_term: 1 leader_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } } }
I20260810 18:41:30.337826 8420 tablet_replica.cc:442] T 6361527cfd3640f39308976247403a55 P 1cc8d7149e8a4a3d898b3043f7a8ba80: This TxnStatusTablet replica's current role is: LEADER
I20260810 18:41:30.338738 8435 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 18:41:30.339071 8435 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 18:41:30.339500 7654 catalog_manager.cc:5495] T 6361527cfd3640f39308976247403a55 P 1cc8d7149e8a4a3d898b3043f7a8ba80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1cc8d7149e8a4a3d898b3043f7a8ba80 (127.4.228.194). New cstate: current_term: 1 leader_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } health_report { overall_health: UNKNOWN } } }
I20260810 18:41:30.359854 8401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:30.352814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50796 (local address 127.4.228.194:43957)
0810 18:41:30.353058 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:30.353066 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:30.353240 (+ 174us) server_negotiation.cc:407] Connection header received
0810 18:41:30.353577 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:30.353582 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:30.353663 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:30.353790 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:41:30.354284 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.355260 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.357238 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.357746 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.357854 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:30.358568 (+ 714us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:41:30.359171 (+ 603us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:41:30.359289 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:30.359455 (+ 166us) server_negotiation.cc:299] Negotiation successful
0810 18:41:30.359594 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260810 18:41:30.359942 8429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:30.352812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:43957 (local address 127.0.0.1:50796)
0810 18:41:30.353096 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:30.353119 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:30.353317 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:30.353854 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:30.353860 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:30.353874 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:41:30.354096 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:30.354108 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.355406 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.355412 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:30.357058 (+ 1646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:30.357076 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.357929 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.357936 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:30.358169 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:30.358251 (+ 82us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:41:30.359289 (+ 1038us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:41:30.359316 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 18:41:30.359423 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 18:41:30.359600 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260810 18:41:30.369751 8294 raft_consensus.cc:1261] T 6361527cfd3640f39308976247403a55 P 06aed71215d44a608962bc2920f6f5dc [term 1 FOLLOWER]: Refusing update from remote peer 1cc8d7149e8a4a3d898b3043f7a8ba80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:41:30.369851 7886 raft_consensus.cc:1261] T 6361527cfd3640f39308976247403a55 P 15095ef6292f44f4a66a82e97497bb20 [term 1 FOLLOWER]: Refusing update from remote peer 1cc8d7149e8a4a3d898b3043f7a8ba80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:41:30.370919 8420 consensus_queue.cc:1035] T 6361527cfd3640f39308976247403a55 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:41:30.371435 8416 consensus_queue.cc:1035] T 6361527cfd3640f39308976247403a55 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:41:30.386379 8412 tablet_replica.cc:439] T 6361527cfd3640f39308976247403a55 P 06aed71215d44a608962bc2920f6f5dc: TxnStatusTablet state changed. Reason: New leader 1cc8d7149e8a4a3d898b3043f7a8ba80. Latest consensus state: current_term: 1 leader_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } } }
I20260810 18:41:30.386776 8412 tablet_replica.cc:442] T 6361527cfd3640f39308976247403a55 P 06aed71215d44a608962bc2920f6f5dc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:41:30.389325 8438 mvcc.cc:205] Tried to move back new op lower bound from 7317042341343068160 to 7317042341217378304. Current Snapshot: MvccSnapshot[applied={T|T < 7317042341343068160}]
I20260810 18:41:30.390544 8420 tablet_replica.cc:439] T 6361527cfd3640f39308976247403a55 P 1cc8d7149e8a4a3d898b3043f7a8ba80: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } } }
I20260810 18:41:30.390858 8420 tablet_replica.cc:442] T 6361527cfd3640f39308976247403a55 P 1cc8d7149e8a4a3d898b3043f7a8ba80: This TxnStatusTablet replica's current role is: LEADER
I20260810 18:41:30.395620 8415 tablet_replica.cc:439] T 6361527cfd3640f39308976247403a55 P 15095ef6292f44f4a66a82e97497bb20: TxnStatusTablet state changed. Reason: New leader 1cc8d7149e8a4a3d898b3043f7a8ba80. Latest consensus state: current_term: 1 leader_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } } }
I20260810 18:41:30.395936 8415 tablet_replica.cc:442] T 6361527cfd3640f39308976247403a55 P 15095ef6292f44f4a66a82e97497bb20: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:41:30.399128 8412 tablet_replica.cc:439] T 6361527cfd3640f39308976247403a55 P 06aed71215d44a608962bc2920f6f5dc: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } } }
I20260810 18:41:30.399444 8412 tablet_replica.cc:442] T 6361527cfd3640f39308976247403a55 P 06aed71215d44a608962bc2920f6f5dc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:41:30.402860 8416 tablet_replica.cc:439] T 6361527cfd3640f39308976247403a55 P 1cc8d7149e8a4a3d898b3043f7a8ba80: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } } }
I20260810 18:41:30.403198 8416 tablet_replica.cc:442] T 6361527cfd3640f39308976247403a55 P 1cc8d7149e8a4a3d898b3043f7a8ba80: This TxnStatusTablet replica's current role is: LEADER
I20260810 18:41:30.407467 8440 mvcc.cc:205] Tried to move back new op lower bound from 7317042341343068160 to 7317042341217378304. Current Snapshot: MvccSnapshot[applied={T|T < 7317042341343068160}]
I20260810 18:41:30.433172 8415 tablet_replica.cc:439] T 6361527cfd3640f39308976247403a55 P 15095ef6292f44f4a66a82e97497bb20: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } } }
I20260810 18:41:30.433485 8415 tablet_replica.cc:442] T 6361527cfd3640f39308976247403a55 P 15095ef6292f44f4a66a82e97497bb20: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:41:30.439579 8401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:30.418194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50800 (local address 127.4.228.194:43957)
0810 18:41:30.418446 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:30.418455 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:30.427728 (+ 9273us) server_negotiation.cc:407] Connection header received
0810 18:41:30.428426 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:30.428436 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:30.428577 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:30.428664 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:41:30.429829 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.431454 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.436832 (+ 5378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.437521 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.437668 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:30.438389 (+ 721us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:41:30.438625 (+ 236us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:41:30.438763 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:30.439069 (+ 306us) server_negotiation.cc:299] Negotiation successful
0810 18:41:30.439186 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260810 18:41:30.460201 8401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:30.452545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:52873 (local address 127.4.228.194:43957)
0810 18:41:30.452789 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:30.452796 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:30.453623 (+ 827us) server_negotiation.cc:407] Connection header received
0810 18:41:30.453802 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:30.453808 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:30.453907 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:30.454016 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:41:30.454714 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.455980 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.457654 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.458438 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.458611 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:30.459203 (+ 592us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:41:30.459381 (+ 178us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:41:30.459501 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:30.459707 (+ 206us) server_negotiation.cc:299] Negotiation successful
0810 18:41:30.459819 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":78}
I20260810 18:41:30.460260 8458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:30.452619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:43957 (local address 127.4.228.194:52873)
0810 18:41:30.453497 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:30.453517 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:30.453666 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:30.454064 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:30.454068 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:30.454078 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:41:30.454546 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:30.454558 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.456159 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.456164 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:30.457489 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:30.457509 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.458694 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.458701 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:30.458869 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:30.458936 (+ 67us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:41:30.459552 (+ 616us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:41:30.459566 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 18:41:30.459655 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 18:41:30.459875 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"thread_start_us":445,"threads_started":1}
I20260810 18:41:30.472054 8294 raft_consensus.cc:1261] T e1fe31c8a9624a62a21540125c0a5a12 P 06aed71215d44a608962bc2920f6f5dc [term 1 FOLLOWER]: Refusing update from remote peer 1cc8d7149e8a4a3d898b3043f7a8ba80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:41:30.472255 7886 raft_consensus.cc:1261] T e1fe31c8a9624a62a21540125c0a5a12 P 15095ef6292f44f4a66a82e97497bb20 [term 1 FOLLOWER]: Refusing update from remote peer 1cc8d7149e8a4a3d898b3043f7a8ba80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:41:30.472925 8448 consensus_queue.cc:1035] T e1fe31c8a9624a62a21540125c0a5a12 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:41:30.473273 8420 consensus_queue.cc:1035] T e1fe31c8a9624a62a21540125c0a5a12 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:41:30.479218 8439 mvcc.cc:205] Tried to move back new op lower bound from 7317042341767716864 to 7317042340074967040. Current Snapshot: MvccSnapshot[applied={T|T < 7317042341767716864}]
I20260810 18:41:30.550863 7651 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:41:30.551180 7651 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:41:30.555467 7651 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a3ca18b5d0844726a51e2587276c30d9: Sending DeleteTablet for 3 replicas of tablet e1fe31c8a9624a62a21540125c0a5a12
I20260810 18:41:30.556917 8059 tablet_service.cc:1514] Processing DeleteTablet for tablet e1fe31c8a9624a62a21540125c0a5a12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:41:30 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:50780
I20260810 18:41:30.556931 7855 tablet_service.cc:1514] Processing DeleteTablet for tablet e1fe31c8a9624a62a21540125c0a5a12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:41:30 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:58080
I20260810 18:41:30.556980 8264 tablet_service.cc:1514] Processing DeleteTablet for tablet e1fe31c8a9624a62a21540125c0a5a12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:41:30 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:55444
I20260810 18:41:30.562664 7654 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35706:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:41:30.563475 7654 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:41:30.568358 8463 tablet_replica.cc:330] T e1fe31c8a9624a62a21540125c0a5a12 P 15095ef6292f44f4a66a82e97497bb20: stopping tablet replica
I20260810 18:41:30.569160 8463 raft_consensus.cc:2229] T e1fe31c8a9624a62a21540125c0a5a12 P 15095ef6292f44f4a66a82e97497bb20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:41:30.569677 8463 raft_consensus.cc:2258] T e1fe31c8a9624a62a21540125c0a5a12 P 15095ef6292f44f4a66a82e97497bb20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:41:30.572930 8463 ts_tablet_manager.cc:1905] T e1fe31c8a9624a62a21540125c0a5a12 P 15095ef6292f44f4a66a82e97497bb20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:41:30.574093 8262 tablet_service.cc:1467] Processing CreateTablet for tablet 745748fb5f9a4fd5be99fe239f328ad5 (DEFAULT_TABLE table=test-table [id=1065d365367a4f43851fa2077c978c58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:41:30.574199 8058 tablet_service.cc:1467] Processing CreateTablet for tablet 745748fb5f9a4fd5be99fe239f328ad5 (DEFAULT_TABLE table=test-table [id=1065d365367a4f43851fa2077c978c58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:41:30.574723 8262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 745748fb5f9a4fd5be99fe239f328ad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:30.574846 8058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 745748fb5f9a4fd5be99fe239f328ad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:30.579102 7855 tablet_service.cc:1467] Processing CreateTablet for tablet 745748fb5f9a4fd5be99fe239f328ad5 (DEFAULT_TABLE table=test-table [id=1065d365367a4f43851fa2077c978c58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:41:30.579772 8462 tablet_replica.cc:330] T e1fe31c8a9624a62a21540125c0a5a12 P 1cc8d7149e8a4a3d898b3043f7a8ba80: stopping tablet replica
I20260810 18:41:30.580428 8462 raft_consensus.cc:2229] T e1fe31c8a9624a62a21540125c0a5a12 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:41:30.581274 8462 raft_consensus.cc:2258] T e1fe31c8a9624a62a21540125c0a5a12 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:41:30.583835 8462 ts_tablet_manager.cc:1905] T e1fe31c8a9624a62a21540125c0a5a12 P 1cc8d7149e8a4a3d898b3043f7a8ba80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:41:30.584340 7855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 745748fb5f9a4fd5be99fe239f328ad5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:30.586061 8403 tablet_bootstrap.cc:492] T 745748fb5f9a4fd5be99fe239f328ad5 P 06aed71215d44a608962bc2920f6f5dc: Bootstrap starting.
I20260810 18:41:30.583910 8405 tablet_bootstrap.cc:492] T 745748fb5f9a4fd5be99fe239f328ad5 P 1cc8d7149e8a4a3d898b3043f7a8ba80: Bootstrap starting.
I20260810 18:41:30.588060 8405 tablet_bootstrap.cc:654] T 745748fb5f9a4fd5be99fe239f328ad5 P 1cc8d7149e8a4a3d898b3043f7a8ba80: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:30.588230 8403 tablet_bootstrap.cc:654] T 745748fb5f9a4fd5be99fe239f328ad5 P 06aed71215d44a608962bc2920f6f5dc: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:30.588950 8465 tablet_replica.cc:330] T e1fe31c8a9624a62a21540125c0a5a12 P 06aed71215d44a608962bc2920f6f5dc: stopping tablet replica
I20260810 18:41:30.589594 8465 raft_consensus.cc:2229] T e1fe31c8a9624a62a21540125c0a5a12 P 06aed71215d44a608962bc2920f6f5dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:41:30.590116 8465 raft_consensus.cc:2258] T e1fe31c8a9624a62a21540125c0a5a12 P 06aed71215d44a608962bc2920f6f5dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:41:30.593350 8404 tablet_bootstrap.cc:492] T 745748fb5f9a4fd5be99fe239f328ad5 P 15095ef6292f44f4a66a82e97497bb20: Bootstrap starting.
I20260810 18:41:30.593931 8465 ts_tablet_manager.cc:1905] T e1fe31c8a9624a62a21540125c0a5a12 P 06aed71215d44a608962bc2920f6f5dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:41:30.595621 8404 tablet_bootstrap.cc:654] T 745748fb5f9a4fd5be99fe239f328ad5 P 15095ef6292f44f4a66a82e97497bb20: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:30.604205 8403 tablet_bootstrap.cc:492] T 745748fb5f9a4fd5be99fe239f328ad5 P 06aed71215d44a608962bc2920f6f5dc: No bootstrap required, opened a new log
I20260810 18:41:30.604242 8405 tablet_bootstrap.cc:492] T 745748fb5f9a4fd5be99fe239f328ad5 P 1cc8d7149e8a4a3d898b3043f7a8ba80: No bootstrap required, opened a new log
I20260810 18:41:30.604395 8403 ts_tablet_manager.cc:1397] T 745748fb5f9a4fd5be99fe239f328ad5 P 06aed71215d44a608962bc2920f6f5dc: Time spent bootstrapping tablet: real 0.018s user 0.007s sys 0.003s
I20260810 18:41:30.604437 8405 ts_tablet_manager.cc:1397] T 745748fb5f9a4fd5be99fe239f328ad5 P 1cc8d7149e8a4a3d898b3043f7a8ba80: Time spent bootstrapping tablet: real 0.021s user 0.006s sys 0.003s
I20260810 18:41:30.605125 8463 ts_tablet_manager.cc:1918] T e1fe31c8a9624a62a21540125c0a5a12 P 15095ef6292f44f4a66a82e97497bb20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:41:30.605185 8403 raft_consensus.cc:348] T 745748fb5f9a4fd5be99fe239f328ad5 P 06aed71215d44a608962bc2920f6f5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:30.605427 8463 log.cc:1196] T e1fe31c8a9624a62a21540125c0a5a12 P 15095ef6292f44f4a66a82e97497bb20: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-0/wal/wals/e1fe31c8a9624a62a21540125c0a5a12
I20260810 18:41:30.605439 8403 raft_consensus.cc:374] T 745748fb5f9a4fd5be99fe239f328ad5 P 06aed71215d44a608962bc2920f6f5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:30.605512 8403 raft_consensus.cc:729] T 745748fb5f9a4fd5be99fe239f328ad5 P 06aed71215d44a608962bc2920f6f5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06aed71215d44a608962bc2920f6f5dc, State: Initialized, Role: FOLLOWER
I20260810 18:41:30.605829 8403 consensus_queue.cc:260] T 745748fb5f9a4fd5be99fe239f328ad5 P 06aed71215d44a608962bc2920f6f5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:30.606209 8463 ts_tablet_manager.cc:1939] T e1fe31c8a9624a62a21540125c0a5a12 P 15095ef6292f44f4a66a82e97497bb20: Deleting consensus metadata
I20260810 18:41:30.606209 8405 raft_consensus.cc:348] T 745748fb5f9a4fd5be99fe239f328ad5 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:30.606611 8405 raft_consensus.cc:374] T 745748fb5f9a4fd5be99fe239f328ad5 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:30.606829 8405 raft_consensus.cc:729] T 745748fb5f9a4fd5be99fe239f328ad5 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc8d7149e8a4a3d898b3043f7a8ba80, State: Initialized, Role: FOLLOWER
I20260810 18:41:30.607323 8405 consensus_queue.cc:260] T 745748fb5f9a4fd5be99fe239f328ad5 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:30.608994 8405 ts_tablet_manager.cc:1428] T 745748fb5f9a4fd5be99fe239f328ad5 P 1cc8d7149e8a4a3d898b3043f7a8ba80: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260810 18:41:30.609149 8403 ts_tablet_manager.cc:1428] T 745748fb5f9a4fd5be99fe239f328ad5 P 06aed71215d44a608962bc2920f6f5dc: Time spent starting tablet: real 0.005s user 0.002s sys 0.001s
I20260810 18:41:30.610626 7635 catalog_manager.cc:4841] TS 15095ef6292f44f4a66a82e97497bb20 (127.4.228.193:43765): tablet e1fe31c8a9624a62a21540125c0a5a12 (table test-table [id=67eb84adb3854c1089f583c6b4f1f435]) successfully deleted
I20260810 18:41:30.612219 8462 ts_tablet_manager.cc:1918] T e1fe31c8a9624a62a21540125c0a5a12 P 1cc8d7149e8a4a3d898b3043f7a8ba80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:41:30.612411 8462 log.cc:1196] T e1fe31c8a9624a62a21540125c0a5a12 P 1cc8d7149e8a4a3d898b3043f7a8ba80: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-1/wal/wals/e1fe31c8a9624a62a21540125c0a5a12
I20260810 18:41:30.612993 8462 ts_tablet_manager.cc:1939] T e1fe31c8a9624a62a21540125c0a5a12 P 1cc8d7149e8a4a3d898b3043f7a8ba80: Deleting consensus metadata
I20260810 18:41:30.614800 7638 catalog_manager.cc:4841] TS 1cc8d7149e8a4a3d898b3043f7a8ba80 (127.4.228.194:43957): tablet e1fe31c8a9624a62a21540125c0a5a12 (table test-table [id=67eb84adb3854c1089f583c6b4f1f435]) successfully deleted
I20260810 18:41:30.616422 8404 tablet_bootstrap.cc:492] T 745748fb5f9a4fd5be99fe239f328ad5 P 15095ef6292f44f4a66a82e97497bb20: No bootstrap required, opened a new log
I20260810 18:41:30.616614 8404 ts_tablet_manager.cc:1397] T 745748fb5f9a4fd5be99fe239f328ad5 P 15095ef6292f44f4a66a82e97497bb20: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.007s
I20260810 18:41:30.617368 8404 raft_consensus.cc:348] T 745748fb5f9a4fd5be99fe239f328ad5 P 15095ef6292f44f4a66a82e97497bb20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:30.617616 8404 raft_consensus.cc:374] T 745748fb5f9a4fd5be99fe239f328ad5 P 15095ef6292f44f4a66a82e97497bb20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:30.617702 8404 raft_consensus.cc:729] T 745748fb5f9a4fd5be99fe239f328ad5 P 15095ef6292f44f4a66a82e97497bb20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15095ef6292f44f4a66a82e97497bb20, State: Initialized, Role: FOLLOWER
I20260810 18:41:30.617954 8404 consensus_queue.cc:260] T 745748fb5f9a4fd5be99fe239f328ad5 P 15095ef6292f44f4a66a82e97497bb20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:30.618057 8465 ts_tablet_manager.cc:1918] T e1fe31c8a9624a62a21540125c0a5a12 P 06aed71215d44a608962bc2920f6f5dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:41:30.618245 8465 log.cc:1196] T e1fe31c8a9624a62a21540125c0a5a12 P 06aed71215d44a608962bc2920f6f5dc: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-2/wal/wals/e1fe31c8a9624a62a21540125c0a5a12
I20260810 18:41:30.618580 8404 ts_tablet_manager.cc:1428] T 745748fb5f9a4fd5be99fe239f328ad5 P 15095ef6292f44f4a66a82e97497bb20: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 18:41:30.618827 8465 ts_tablet_manager.cc:1939] T e1fe31c8a9624a62a21540125c0a5a12 P 06aed71215d44a608962bc2920f6f5dc: Deleting consensus metadata
I20260810 18:41:30.620801 7639 catalog_manager.cc:4841] TS 06aed71215d44a608962bc2920f6f5dc (127.4.228.195:45901): tablet e1fe31c8a9624a62a21540125c0a5a12 (table test-table [id=67eb84adb3854c1089f583c6b4f1f435]) successfully deleted
I20260810 18:41:30.648573 8412 raft_consensus.cc:482] T 745748fb5f9a4fd5be99fe239f328ad5 P 06aed71215d44a608962bc2920f6f5dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:41:30.648890 8412 raft_consensus.cc:504] T 745748fb5f9a4fd5be99fe239f328ad5 P 06aed71215d44a608962bc2920f6f5dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:30.650780 8412 leader_election.cc:288] T 745748fb5f9a4fd5be99fe239f328ad5 P 06aed71215d44a608962bc2920f6f5dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15095ef6292f44f4a66a82e97497bb20 (127.4.228.193:43765), 1cc8d7149e8a4a3d898b3043f7a8ba80 (127.4.228.194:43957)
I20260810 18:41:30.671227 8476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:30.651967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:52487 (local address 127.4.228.193:43765)
0810 18:41:30.664665 (+ 12698us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:30.664674 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:30.664696 (+ 22us) server_negotiation.cc:407] Connection header received
0810 18:41:30.664778 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:30.664783 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:30.664864 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:30.664969 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:41:30.665749 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.666920 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.669158 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.670084 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.670284 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:30.670372 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:30.670726 (+ 354us) server_negotiation.cc:299] Negotiation successful
0810 18:41:30.670876 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12538,"thread_start_us":12424,"threads_started":1}
I20260810 18:41:30.671300 8472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:30.650987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:43765 (local address 127.4.228.195:52487)
0810 18:41:30.663211 (+ 12224us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:30.663239 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:30.663427 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:30.665188 (+ 1761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:30.665194 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:30.665225 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:41:30.665508 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:30.665518 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.667083 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.667090 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:30.668986 (+ 1896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:30.669001 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.670274 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.670281 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:30.670539 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:30.670573 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 18:41:30.670689 (+ 116us) client_negotiation.cc:240] Negotiation successful
0810 18:41:30.670842 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12037,"mutex_wait_us":25,"thread_start_us":12534,"threads_started":1}
I20260810 18:41:30.672365 7886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "745748fb5f9a4fd5be99fe239f328ad5" candidate_uuid: "06aed71215d44a608962bc2920f6f5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15095ef6292f44f4a66a82e97497bb20" is_pre_election: true
I20260810 18:41:30.672662 7886 raft_consensus.cc:2454] T 745748fb5f9a4fd5be99fe239f328ad5 P 15095ef6292f44f4a66a82e97497bb20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06aed71215d44a608962bc2920f6f5dc in term 0.
I20260810 18:41:30.672863 8401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:30.651469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:47803 (local address 127.4.228.194:43957)
0810 18:41:30.651736 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:30.651746 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:30.665140 (+ 13394us) server_negotiation.cc:407] Connection header received
0810 18:41:30.665547 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:30.665554 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:30.665645 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:30.665720 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:41:30.666449 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.667678 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.670279 (+ 2601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.671706 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.671864 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:30.671938 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:30.672389 (+ 451us) server_negotiation.cc:299] Negotiation successful
0810 18:41:30.672537 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260810 18:41:30.672933 8474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:30.651114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:43957 (local address 127.4.228.195:47803)
0810 18:41:30.665002 (+ 13888us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:30.665025 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:30.665294 (+ 269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:30.665945 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:30.665949 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:30.665971 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:41:30.666267 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:30.666274 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.667882 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.667887 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:30.670100 (+ 2213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:30.670116 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.672005 (+ 1889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.672013 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:30.672207 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:30.672234 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 18:41:30.672358 (+ 124us) client_negotiation.cc:240] Negotiation successful
0810 18:41:30.672493 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13753,"mutex_wait_us":29,"thread_start_us":14465,"threads_started":1}
I20260810 18:41:30.673498 8190 leader_election.cc:302] T 745748fb5f9a4fd5be99fe239f328ad5 P 06aed71215d44a608962bc2920f6f5dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06aed71215d44a608962bc2920f6f5dc, 15095ef6292f44f4a66a82e97497bb20; no voters:
I20260810 18:41:30.674041 8412 raft_consensus.cc:2781] T 745748fb5f9a4fd5be99fe239f328ad5 P 06aed71215d44a608962bc2920f6f5dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:41:30.674193 8412 raft_consensus.cc:482] T 745748fb5f9a4fd5be99fe239f328ad5 P 06aed71215d44a608962bc2920f6f5dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:41:30.674042 8092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "745748fb5f9a4fd5be99fe239f328ad5" candidate_uuid: "06aed71215d44a608962bc2920f6f5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" is_pre_election: true
I20260810 18:41:30.674314 8412 raft_consensus.cc:3037] T 745748fb5f9a4fd5be99fe239f328ad5 P 06aed71215d44a608962bc2920f6f5dc [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:30.674607 8092 raft_consensus.cc:2454] T 745748fb5f9a4fd5be99fe239f328ad5 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06aed71215d44a608962bc2920f6f5dc in term 0.
I20260810 18:41:30.677265 8412 raft_consensus.cc:504] T 745748fb5f9a4fd5be99fe239f328ad5 P 06aed71215d44a608962bc2920f6f5dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:30.678086 8412 leader_election.cc:288] T 745748fb5f9a4fd5be99fe239f328ad5 P 06aed71215d44a608962bc2920f6f5dc [CANDIDATE]: Term 1 election: Requested vote from peers 15095ef6292f44f4a66a82e97497bb20 (127.4.228.193:43765), 1cc8d7149e8a4a3d898b3043f7a8ba80 (127.4.228.194:43957)
I20260810 18:41:30.678567 7886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "745748fb5f9a4fd5be99fe239f328ad5" candidate_uuid: "06aed71215d44a608962bc2920f6f5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15095ef6292f44f4a66a82e97497bb20"
I20260810 18:41:30.678660 8092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "745748fb5f9a4fd5be99fe239f328ad5" candidate_uuid: "06aed71215d44a608962bc2920f6f5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80"
I20260810 18:41:30.678817 7886 raft_consensus.cc:3037] T 745748fb5f9a4fd5be99fe239f328ad5 P 15095ef6292f44f4a66a82e97497bb20 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:30.678864 8092 raft_consensus.cc:3037] T 745748fb5f9a4fd5be99fe239f328ad5 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:30.680963 7886 raft_consensus.cc:2454] T 745748fb5f9a4fd5be99fe239f328ad5 P 15095ef6292f44f4a66a82e97497bb20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06aed71215d44a608962bc2920f6f5dc in term 1.
I20260810 18:41:30.681427 8190 leader_election.cc:302] T 745748fb5f9a4fd5be99fe239f328ad5 P 06aed71215d44a608962bc2920f6f5dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06aed71215d44a608962bc2920f6f5dc, 15095ef6292f44f4a66a82e97497bb20; no voters:
I20260810 18:41:30.681759 8412 raft_consensus.cc:2781] T 745748fb5f9a4fd5be99fe239f328ad5 P 06aed71215d44a608962bc2920f6f5dc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:41:30.681905 8092 raft_consensus.cc:2454] T 745748fb5f9a4fd5be99fe239f328ad5 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06aed71215d44a608962bc2920f6f5dc in term 1.
I20260810 18:41:30.682343 8412 raft_consensus.cc:686] T 745748fb5f9a4fd5be99fe239f328ad5 P 06aed71215d44a608962bc2920f6f5dc [term 1 LEADER]: Becoming Leader. State: Replica: 06aed71215d44a608962bc2920f6f5dc, State: Running, Role: LEADER
I20260810 18:41:30.682848 8412 consensus_queue.cc:237] T 745748fb5f9a4fd5be99fe239f328ad5 P 06aed71215d44a608962bc2920f6f5dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:30.687738 7651 catalog_manager.cc:5495] T 745748fb5f9a4fd5be99fe239f328ad5 P 06aed71215d44a608962bc2920f6f5dc reported cstate change: term changed from 0 to 1, leader changed from <none> to 06aed71215d44a608962bc2920f6f5dc (127.4.228.195). New cstate: current_term: 1 leader_uuid: "06aed71215d44a608962bc2920f6f5dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } health_report { overall_health: HEALTHY } } }
I20260810 18:41:30.734555 8480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:30.726970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55454 (local address 127.4.228.195:45901)
0810 18:41:30.727677 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:30.727681 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:30.727702 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:41:30.727793 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:30.727797 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:30.727876 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:30.728032 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:41:30.728569 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.729791 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.731464 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.732127 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.732274 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:30.732838 (+ 564us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:41:30.733668 (+ 830us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:41:30.733843 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:30.733960 (+ 117us) server_negotiation.cc:299] Negotiation successful
0810 18:41:30.734065 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":419,"threads_started":1}
I20260810 18:41:30.754004 8484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:30.747222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:43957 (local address 127.4.228.195:55845)
0810 18:41:30.748132 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:30.748151 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:30.748294 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:30.748758 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:30.748762 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:30.748773 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:41:30.749080 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:30.749090 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.750595 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.750605 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:30.751709 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:30.751722 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.752794 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.752799 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:30.752938 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:30.753021 (+ 83us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:41:30.753490 (+ 469us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:41:30.753500 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 18:41:30.753570 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 18:41:30.753688 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":740,"thread_start_us":477,"threads_started":1}
I20260810 18:41:30.754066 8401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:30.747234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:55845 (local address 127.4.228.194:43957)
0810 18:41:30.747478 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:30.747486 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:30.748256 (+ 770us) server_negotiation.cc:407] Connection header received
0810 18:41:30.748442 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:30.748448 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:30.748584 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:30.748709 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:41:30.749251 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.750427 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.751875 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.752582 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.752742 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:30.753217 (+ 475us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:41:30.753377 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:41:30.753479 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:30.753624 (+ 145us) server_negotiation.cc:299] Negotiation successful
0810 18:41:30.753718 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 18:41:30.764808 7886 raft_consensus.cc:1261] T 745748fb5f9a4fd5be99fe239f328ad5 P 15095ef6292f44f4a66a82e97497bb20 [term 1 FOLLOWER]: Refusing update from remote peer 06aed71215d44a608962bc2920f6f5dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:41:30.765115 8092 raft_consensus.cc:1261] T 745748fb5f9a4fd5be99fe239f328ad5 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 1 FOLLOWER]: Refusing update from remote peer 06aed71215d44a608962bc2920f6f5dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:41:30.765611 8477 consensus_queue.cc:1035] T 745748fb5f9a4fd5be99fe239f328ad5 P 06aed71215d44a608962bc2920f6f5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:41:30.766089 8412 consensus_queue.cc:1035] T 745748fb5f9a4fd5be99fe239f328ad5 P 06aed71215d44a608962bc2920f6f5dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:41:30.807240 8458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:30.799810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:45901 (local address 127.4.228.194:35253)
0810 18:41:30.800221 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:30.800245 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:30.800368 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:30.801330 (+ 962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:30.801341 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:30.801354 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:41:30.801552 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:30.801565 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.803121 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.803127 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:30.804873 (+ 1746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:30.804891 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.805814 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.805820 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:30.806024 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:30.806086 (+ 62us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:41:30.806616 (+ 530us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:41:30.806638 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 18:41:30.806722 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 18:41:30.806897 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231}
I20260810 18:41:30.807904 8480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:30.800631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:35253 (local address 127.4.228.195:45901)
0810 18:41:30.800824 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:30.800830 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:30.800851 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:41:30.800925 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:30.800930 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:30.801022 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:30.801132 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:41:30.801756 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.802944 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.805039 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:30.805608 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:30.805757 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:30.806299 (+ 542us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:41:30.806441 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:41:30.806562 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:30.807531 (+ 969us) server_negotiation.cc:299] Negotiation successful
0810 18:41:30.807619 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260810 18:41:30.809331 8439 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 18:41:32.450745 7654 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:41:32.451179 7654 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:41:32.457782 7654 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a3ca18b5d0844726a51e2587276c30d9: Sending DeleteTablet for 3 replicas of tablet 745748fb5f9a4fd5be99fe239f328ad5
I20260810 18:41:32.459127 7855 tablet_service.cc:1514] Processing DeleteTablet for tablet 745748fb5f9a4fd5be99fe239f328ad5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:41:32 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:58080
I20260810 18:41:32.459403 8059 tablet_service.cc:1514] Processing DeleteTablet for tablet 745748fb5f9a4fd5be99fe239f328ad5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:41:32 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:50780
I20260810 18:41:32.459425 8264 tablet_service.cc:1514] Processing DeleteTablet for tablet 745748fb5f9a4fd5be99fe239f328ad5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:41:32 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:55444
I20260810 18:41:32.472376 8532 tablet_replica.cc:330] T 745748fb5f9a4fd5be99fe239f328ad5 P 06aed71215d44a608962bc2920f6f5dc: stopping tablet replica
I20260810 18:41:32.473490 8532 raft_consensus.cc:2229] T 745748fb5f9a4fd5be99fe239f328ad5 P 06aed71215d44a608962bc2920f6f5dc [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:41:32.474767 8532 raft_consensus.cc:2258] T 745748fb5f9a4fd5be99fe239f328ad5 P 06aed71215d44a608962bc2920f6f5dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:41:32.475502 8530 tablet_replica.cc:330] T 745748fb5f9a4fd5be99fe239f328ad5 P 1cc8d7149e8a4a3d898b3043f7a8ba80: stopping tablet replica
I20260810 18:41:32.476235 8530 raft_consensus.cc:2229] T 745748fb5f9a4fd5be99fe239f328ad5 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:41:32.476378 8532 ts_tablet_manager.cc:1905] T 745748fb5f9a4fd5be99fe239f328ad5 P 06aed71215d44a608962bc2920f6f5dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:41:32.476521 8528 tablet_replica.cc:330] T 745748fb5f9a4fd5be99fe239f328ad5 P 15095ef6292f44f4a66a82e97497bb20: stopping tablet replica
I20260810 18:41:32.476783 8530 raft_consensus.cc:2258] T 745748fb5f9a4fd5be99fe239f328ad5 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:41:32.476919 8528 raft_consensus.cc:2229] T 745748fb5f9a4fd5be99fe239f328ad5 P 15095ef6292f44f4a66a82e97497bb20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:41:32.477361 8528 raft_consensus.cc:2258] T 745748fb5f9a4fd5be99fe239f328ad5 P 15095ef6292f44f4a66a82e97497bb20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:41:32.478982 8530 ts_tablet_manager.cc:1905] T 745748fb5f9a4fd5be99fe239f328ad5 P 1cc8d7149e8a4a3d898b3043f7a8ba80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:41:32.479245 8528 ts_tablet_manager.cc:1905] T 745748fb5f9a4fd5be99fe239f328ad5 P 15095ef6292f44f4a66a82e97497bb20: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 10 18:41:32 dist-test-slave-7kzw krb5kdc[7591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387292, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
I20260810 18:41:32.581815 8528 ts_tablet_manager.cc:1918] T 745748fb5f9a4fd5be99fe239f328ad5 P 15095ef6292f44f4a66a82e97497bb20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 18:41:32.582093 8528 log.cc:1196] T 745748fb5f9a4fd5be99fe239f328ad5 P 15095ef6292f44f4a66a82e97497bb20: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-0/wal/wals/745748fb5f9a4fd5be99fe239f328ad5
I20260810 18:41:32.582132 8532 ts_tablet_manager.cc:1918] T 745748fb5f9a4fd5be99fe239f328ad5 P 06aed71215d44a608962bc2920f6f5dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 18:41:32.582371 8532 log.cc:1196] T 745748fb5f9a4fd5be99fe239f328ad5 P 06aed71215d44a608962bc2920f6f5dc: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-2/wal/wals/745748fb5f9a4fd5be99fe239f328ad5
I20260810 18:41:32.583084 8528 ts_tablet_manager.cc:1939] T 745748fb5f9a4fd5be99fe239f328ad5 P 15095ef6292f44f4a66a82e97497bb20: Deleting consensus metadata
I20260810 18:41:32.583123 8532 ts_tablet_manager.cc:1939] T 745748fb5f9a4fd5be99fe239f328ad5 P 06aed71215d44a608962bc2920f6f5dc: Deleting consensus metadata
W20260810 18:41:32.583686 5011 mini_kdc.cc:317] Time spent kinit for test-user: real 0.122s user 0.004s sys 0.016s
I20260810 18:41:32.586429 8530 ts_tablet_manager.cc:1918] T 745748fb5f9a4fd5be99fe239f328ad5 P 1cc8d7149e8a4a3d898b3043f7a8ba80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 18:41:32.586714 8530 log.cc:1196] T 745748fb5f9a4fd5be99fe239f328ad5 P 1cc8d7149e8a4a3d898b3043f7a8ba80: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-1/wal/wals/745748fb5f9a4fd5be99fe239f328ad5
I20260810 18:41:32.587147 8530 ts_tablet_manager.cc:1939] T 745748fb5f9a4fd5be99fe239f328ad5 P 1cc8d7149e8a4a3d898b3043f7a8ba80: Deleting consensus metadata
I20260810 18:41:32.591120 7639 catalog_manager.cc:4841] TS 06aed71215d44a608962bc2920f6f5dc (127.4.228.195:45901): tablet 745748fb5f9a4fd5be99fe239f328ad5 (table test-table [id=1065d365367a4f43851fa2077c978c58]) successfully deleted
I20260810 18:41:32.591984 7635 catalog_manager.cc:4841] TS 15095ef6292f44f4a66a82e97497bb20 (127.4.228.193:43765): tablet 745748fb5f9a4fd5be99fe239f328ad5 (table test-table [id=1065d365367a4f43851fa2077c978c58]) successfully deleted
Aug 10 18:41:32 dist-test-slave-7kzw krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387292, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:32.610821 8543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:32.589141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35720 (local address 127.4.228.254:44281)
0810 18:41:32.589885 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:32.589894 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:32.589979 (+ 85us) server_negotiation.cc:407] Connection header received
0810 18:41:32.590209 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:32.590214 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:32.590298 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:32.590436 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:32.593188 (+ 2752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:32.594008 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:32.595835 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:32.596298 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:32.596414 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:32.602937 (+ 6523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:32.602945 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:32.602948 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:32.602996 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:32.607329 (+ 4333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:32.608101 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:32.608108 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:32.608117 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:32.608341 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:32.608811 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:32.608815 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:32.608817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:32.609202 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:32.609591 (+ 389us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:32.609869 (+ 278us) server_negotiation.cc:299] Negotiation successful
0810 18:41:32.610251 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":424,"threads_started":1}
I20260810 18:41:32.616115 7654 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35720:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:41:32.616922 7654 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:41:32.625854 8059 tablet_service.cc:1467] Processing CreateTablet for tablet 9257ca46dbb945bbbf4a37efd1512ec3 (DEFAULT_TABLE table=test-table [id=74de47ff46df4ac082cefe4cd2acfaa0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:41:32.626523 8059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9257ca46dbb945bbbf4a37efd1512ec3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:32.627449 8264 tablet_service.cc:1467] Processing CreateTablet for tablet 9257ca46dbb945bbbf4a37efd1512ec3 (DEFAULT_TABLE table=test-table [id=74de47ff46df4ac082cefe4cd2acfaa0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:41:32.627894 7855 tablet_service.cc:1467] Processing CreateTablet for tablet 9257ca46dbb945bbbf4a37efd1512ec3 (DEFAULT_TABLE table=test-table [id=74de47ff46df4ac082cefe4cd2acfaa0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:41:32.628080 8264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9257ca46dbb945bbbf4a37efd1512ec3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:32.628489 7855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9257ca46dbb945bbbf4a37efd1512ec3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:32.636451 8544 tablet_bootstrap.cc:492] T 9257ca46dbb945bbbf4a37efd1512ec3 P 1cc8d7149e8a4a3d898b3043f7a8ba80: Bootstrap starting.
I20260810 18:41:32.640125 8544 tablet_bootstrap.cc:654] T 9257ca46dbb945bbbf4a37efd1512ec3 P 1cc8d7149e8a4a3d898b3043f7a8ba80: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:32.644585 8545 tablet_bootstrap.cc:492] T 9257ca46dbb945bbbf4a37efd1512ec3 P 06aed71215d44a608962bc2920f6f5dc: Bootstrap starting.
I20260810 18:41:32.645967 8547 tablet_bootstrap.cc:492] T 9257ca46dbb945bbbf4a37efd1512ec3 P 15095ef6292f44f4a66a82e97497bb20: Bootstrap starting.
I20260810 18:41:32.646844 8544 tablet_bootstrap.cc:492] T 9257ca46dbb945bbbf4a37efd1512ec3 P 1cc8d7149e8a4a3d898b3043f7a8ba80: No bootstrap required, opened a new log
I20260810 18:41:32.647082 8544 ts_tablet_manager.cc:1397] T 9257ca46dbb945bbbf4a37efd1512ec3 P 1cc8d7149e8a4a3d898b3043f7a8ba80: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.009s
I20260810 18:41:32.648089 8545 tablet_bootstrap.cc:654] T 9257ca46dbb945bbbf4a37efd1512ec3 P 06aed71215d44a608962bc2920f6f5dc: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:32.649035 8544 raft_consensus.cc:348] T 9257ca46dbb945bbbf4a37efd1512ec3 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:32.649560 8547 tablet_bootstrap.cc:654] T 9257ca46dbb945bbbf4a37efd1512ec3 P 15095ef6292f44f4a66a82e97497bb20: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:32.649737 8544 raft_consensus.cc:374] T 9257ca46dbb945bbbf4a37efd1512ec3 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:32.650082 8544 raft_consensus.cc:729] T 9257ca46dbb945bbbf4a37efd1512ec3 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc8d7149e8a4a3d898b3043f7a8ba80, State: Initialized, Role: FOLLOWER
I20260810 18:41:32.650846 8544 consensus_queue.cc:260] T 9257ca46dbb945bbbf4a37efd1512ec3 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:32.652688 8545 tablet_bootstrap.cc:492] T 9257ca46dbb945bbbf4a37efd1512ec3 P 06aed71215d44a608962bc2920f6f5dc: No bootstrap required, opened a new log
I20260810 18:41:32.652745 8544 ts_tablet_manager.cc:1428] T 9257ca46dbb945bbbf4a37efd1512ec3 P 1cc8d7149e8a4a3d898b3043f7a8ba80: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260810 18:41:32.652985 8545 ts_tablet_manager.cc:1397] T 9257ca46dbb945bbbf4a37efd1512ec3 P 06aed71215d44a608962bc2920f6f5dc: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260810 18:41:32.653995 8547 tablet_bootstrap.cc:492] T 9257ca46dbb945bbbf4a37efd1512ec3 P 15095ef6292f44f4a66a82e97497bb20: No bootstrap required, opened a new log
I20260810 18:41:32.654230 8547 ts_tablet_manager.cc:1397] T 9257ca46dbb945bbbf4a37efd1512ec3 P 15095ef6292f44f4a66a82e97497bb20: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s
I20260810 18:41:32.654881 8545 raft_consensus.cc:348] T 9257ca46dbb945bbbf4a37efd1512ec3 P 06aed71215d44a608962bc2920f6f5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:32.655434 8545 raft_consensus.cc:374] T 9257ca46dbb945bbbf4a37efd1512ec3 P 06aed71215d44a608962bc2920f6f5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:32.655669 8545 raft_consensus.cc:729] T 9257ca46dbb945bbbf4a37efd1512ec3 P 06aed71215d44a608962bc2920f6f5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06aed71215d44a608962bc2920f6f5dc, State: Initialized, Role: FOLLOWER
I20260810 18:41:32.655740 8547 raft_consensus.cc:348] T 9257ca46dbb945bbbf4a37efd1512ec3 P 15095ef6292f44f4a66a82e97497bb20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:32.656410 8547 raft_consensus.cc:374] T 9257ca46dbb945bbbf4a37efd1512ec3 P 15095ef6292f44f4a66a82e97497bb20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:32.656327 8545 consensus_queue.cc:260] T 9257ca46dbb945bbbf4a37efd1512ec3 P 06aed71215d44a608962bc2920f6f5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:32.656682 8547 raft_consensus.cc:729] T 9257ca46dbb945bbbf4a37efd1512ec3 P 15095ef6292f44f4a66a82e97497bb20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15095ef6292f44f4a66a82e97497bb20, State: Initialized, Role: FOLLOWER
I20260810 18:41:32.657078 8547 consensus_queue.cc:260] T 9257ca46dbb945bbbf4a37efd1512ec3 P 15095ef6292f44f4a66a82e97497bb20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:32.657491 8545 ts_tablet_manager.cc:1428] T 9257ca46dbb945bbbf4a37efd1512ec3 P 06aed71215d44a608962bc2920f6f5dc: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260810 18:41:32.658607 8547 ts_tablet_manager.cc:1428] T 9257ca46dbb945bbbf4a37efd1512ec3 P 15095ef6292f44f4a66a82e97497bb20: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260810 18:41:32.675081 7638 catalog_manager.cc:4841] TS 1cc8d7149e8a4a3d898b3043f7a8ba80 (127.4.228.194:43957): tablet 745748fb5f9a4fd5be99fe239f328ad5 (table test-table [id=1065d365367a4f43851fa2077c978c58]) successfully deleted
I20260810 18:41:32.755535 8500 raft_consensus.cc:482] T 9257ca46dbb945bbbf4a37efd1512ec3 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:41:32.755931 8500 raft_consensus.cc:504] T 9257ca46dbb945bbbf4a37efd1512ec3 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:32.757088 8500 leader_election.cc:288] T 9257ca46dbb945bbbf4a37efd1512ec3 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15095ef6292f44f4a66a82e97497bb20 (127.4.228.193:43765), 06aed71215d44a608962bc2920f6f5dc (127.4.228.195:45901)
I20260810 18:41:32.757897 8293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9257ca46dbb945bbbf4a37efd1512ec3" candidate_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06aed71215d44a608962bc2920f6f5dc" is_pre_election: true
I20260810 18:41:32.758170 7886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9257ca46dbb945bbbf4a37efd1512ec3" candidate_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15095ef6292f44f4a66a82e97497bb20" is_pre_election: true
I20260810 18:41:32.758332 8293 raft_consensus.cc:2454] T 9257ca46dbb945bbbf4a37efd1512ec3 P 06aed71215d44a608962bc2920f6f5dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cc8d7149e8a4a3d898b3043f7a8ba80 in term 0.
I20260810 18:41:32.758586 7886 raft_consensus.cc:2454] T 9257ca46dbb945bbbf4a37efd1512ec3 P 15095ef6292f44f4a66a82e97497bb20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cc8d7149e8a4a3d898b3043f7a8ba80 in term 0.
I20260810 18:41:32.759233 7994 leader_election.cc:302] T 9257ca46dbb945bbbf4a37efd1512ec3 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06aed71215d44a608962bc2920f6f5dc, 1cc8d7149e8a4a3d898b3043f7a8ba80; no voters:
I20260810 18:41:32.759732 8500 raft_consensus.cc:2781] T 9257ca46dbb945bbbf4a37efd1512ec3 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:41:32.760010 8500 raft_consensus.cc:482] T 9257ca46dbb945bbbf4a37efd1512ec3 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:41:32.760236 8500 raft_consensus.cc:3037] T 9257ca46dbb945bbbf4a37efd1512ec3 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:32.781776 8500 raft_consensus.cc:504] T 9257ca46dbb945bbbf4a37efd1512ec3 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:32.782996 8500 leader_election.cc:288] T 9257ca46dbb945bbbf4a37efd1512ec3 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [CANDIDATE]: Term 1 election: Requested vote from peers 15095ef6292f44f4a66a82e97497bb20 (127.4.228.193:43765), 06aed71215d44a608962bc2920f6f5dc (127.4.228.195:45901)
I20260810 18:41:32.783934 8293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9257ca46dbb945bbbf4a37efd1512ec3" candidate_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06aed71215d44a608962bc2920f6f5dc"
I20260810 18:41:32.783957 7886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9257ca46dbb945bbbf4a37efd1512ec3" candidate_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15095ef6292f44f4a66a82e97497bb20"
I20260810 18:41:32.784323 8293 raft_consensus.cc:3037] T 9257ca46dbb945bbbf4a37efd1512ec3 P 06aed71215d44a608962bc2920f6f5dc [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:32.784368 7886 raft_consensus.cc:3037] T 9257ca46dbb945bbbf4a37efd1512ec3 P 15095ef6292f44f4a66a82e97497bb20 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:32.882586 7886 raft_consensus.cc:2454] T 9257ca46dbb945bbbf4a37efd1512ec3 P 15095ef6292f44f4a66a82e97497bb20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cc8d7149e8a4a3d898b3043f7a8ba80 in term 1.
I20260810 18:41:32.882706 8293 raft_consensus.cc:2454] T 9257ca46dbb945bbbf4a37efd1512ec3 P 06aed71215d44a608962bc2920f6f5dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cc8d7149e8a4a3d898b3043f7a8ba80 in term 1.
I20260810 18:41:32.883502 7991 leader_election.cc:302] T 9257ca46dbb945bbbf4a37efd1512ec3 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15095ef6292f44f4a66a82e97497bb20, 1cc8d7149e8a4a3d898b3043f7a8ba80; no voters:
I20260810 18:41:32.884196 8500 raft_consensus.cc:2781] T 9257ca46dbb945bbbf4a37efd1512ec3 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:41:32.884531 8500 raft_consensus.cc:686] T 9257ca46dbb945bbbf4a37efd1512ec3 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 1 LEADER]: Becoming Leader. State: Replica: 1cc8d7149e8a4a3d898b3043f7a8ba80, State: Running, Role: LEADER
I20260810 18:41:32.885075 8500 consensus_queue.cc:237] T 9257ca46dbb945bbbf4a37efd1512ec3 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:32.891054 7651 catalog_manager.cc:5495] T 9257ca46dbb945bbbf4a37efd1512ec3 P 1cc8d7149e8a4a3d898b3043f7a8ba80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1cc8d7149e8a4a3d898b3043f7a8ba80 (127.4.228.194). New cstate: current_term: 1 leader_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } health_report { overall_health: UNKNOWN } } }
I20260810 18:41:32.960167 8568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:32.952010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50814 (local address 127.4.228.194:43957)
0810 18:41:32.953074 (+ 1064us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:32.953080 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:32.953103 (+ 23us) server_negotiation.cc:407] Connection header received
0810 18:41:32.953283 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:32.953287 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:32.953359 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:32.953470 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:41:32.954099 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:32.955294 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:32.956929 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:32.957621 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:32.957769 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:32.958965 (+ 1196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:41:32.959201 (+ 236us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:41:32.959313 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:32.959471 (+ 158us) server_negotiation.cc:299] Negotiation successful
0810 18:41:32.959559 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":858,"thread_start_us":611,"threads_started":1}
I20260810 18:41:32.972599 8293 raft_consensus.cc:1261] T 9257ca46dbb945bbbf4a37efd1512ec3 P 06aed71215d44a608962bc2920f6f5dc [term 1 FOLLOWER]: Refusing update from remote peer 1cc8d7149e8a4a3d898b3043f7a8ba80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:41:32.972600 7886 raft_consensus.cc:1261] T 9257ca46dbb945bbbf4a37efd1512ec3 P 15095ef6292f44f4a66a82e97497bb20 [term 1 FOLLOWER]: Refusing update from remote peer 1cc8d7149e8a4a3d898b3043f7a8ba80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:41:32.973284 8500 consensus_queue.cc:1035] T 9257ca46dbb945bbbf4a37efd1512ec3 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:41:32.973726 8561 consensus_queue.cc:1035] T 9257ca46dbb945bbbf4a37efd1512ec3 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:41:33.042613 7654 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:41:33.042908 7654 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:41:33.046666 7654 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a3ca18b5d0844726a51e2587276c30d9: Sending DeleteTablet for 3 replicas of tablet 9257ca46dbb945bbbf4a37efd1512ec3
I20260810 18:41:33.047971 7855 tablet_service.cc:1514] Processing DeleteTablet for tablet 9257ca46dbb945bbbf4a37efd1512ec3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:41:33 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:58080
I20260810 18:41:33.048099 8264 tablet_service.cc:1514] Processing DeleteTablet for tablet 9257ca46dbb945bbbf4a37efd1512ec3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:41:33 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:55444
I20260810 18:41:33.048454 8532 tablet_replica.cc:330] T 9257ca46dbb945bbbf4a37efd1512ec3 P 06aed71215d44a608962bc2920f6f5dc: stopping tablet replica
I20260810 18:41:33.048454 8528 tablet_replica.cc:330] T 9257ca46dbb945bbbf4a37efd1512ec3 P 15095ef6292f44f4a66a82e97497bb20: stopping tablet replica
I20260810 18:41:33.048580 8059 tablet_service.cc:1514] Processing DeleteTablet for tablet 9257ca46dbb945bbbf4a37efd1512ec3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:41:33 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:50780
I20260810 18:41:33.048761 8532 raft_consensus.cc:2229] T 9257ca46dbb945bbbf4a37efd1512ec3 P 06aed71215d44a608962bc2920f6f5dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:41:33.048974 8528 raft_consensus.cc:2229] T 9257ca46dbb945bbbf4a37efd1512ec3 P 15095ef6292f44f4a66a82e97497bb20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:41:33.049090 8532 raft_consensus.cc:2258] T 9257ca46dbb945bbbf4a37efd1512ec3 P 06aed71215d44a608962bc2920f6f5dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:41:33.049146 8530 tablet_replica.cc:330] T 9257ca46dbb945bbbf4a37efd1512ec3 P 1cc8d7149e8a4a3d898b3043f7a8ba80: stopping tablet replica
I20260810 18:41:33.049306 8528 raft_consensus.cc:2258] T 9257ca46dbb945bbbf4a37efd1512ec3 P 15095ef6292f44f4a66a82e97497bb20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:41:33.049484 8530 raft_consensus.cc:2229] T 9257ca46dbb945bbbf4a37efd1512ec3 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:41:33.049676 7654 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35720:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 18:41:33.049966 8530 raft_consensus.cc:2258] T 9257ca46dbb945bbbf4a37efd1512ec3 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:41:33.050432 8528 ts_tablet_manager.cc:1905] T 9257ca46dbb945bbbf4a37efd1512ec3 P 15095ef6292f44f4a66a82e97497bb20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:41:33.050441 8532 ts_tablet_manager.cc:1905] T 9257ca46dbb945bbbf4a37efd1512ec3 P 06aed71215d44a608962bc2920f6f5dc: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 18:41:33.050731 7654 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:41:33.051824 8530 ts_tablet_manager.cc:1905] T 9257ca46dbb945bbbf4a37efd1512ec3 P 1cc8d7149e8a4a3d898b3043f7a8ba80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:41:33.059890 7855 tablet_service.cc:1467] Processing CreateTablet for tablet 8a8255f74b2a483fae3bfd9113829865 (DEFAULT_TABLE table=test-table [id=fc1d3f21a2f1402f9b83ba469a5ffa2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:41:33.060264 8059 tablet_service.cc:1467] Processing CreateTablet for tablet 8a8255f74b2a483fae3bfd9113829865 (DEFAULT_TABLE table=test-table [id=fc1d3f21a2f1402f9b83ba469a5ffa2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:41:33.060576 8264 tablet_service.cc:1467] Processing CreateTablet for tablet 8a8255f74b2a483fae3bfd9113829865 (DEFAULT_TABLE table=test-table [id=fc1d3f21a2f1402f9b83ba469a5ffa2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:41:33.060971 8059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a8255f74b2a483fae3bfd9113829865. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:33.061173 7855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a8255f74b2a483fae3bfd9113829865. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:33.066144 8530 ts_tablet_manager.cc:1918] T 9257ca46dbb945bbbf4a37efd1512ec3 P 1cc8d7149e8a4a3d898b3043f7a8ba80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:41:33.066427 8530 log.cc:1196] T 9257ca46dbb945bbbf4a37efd1512ec3 P 1cc8d7149e8a4a3d898b3043f7a8ba80: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-1/wal/wals/9257ca46dbb945bbbf4a37efd1512ec3
I20260810 18:41:33.067217 8530 ts_tablet_manager.cc:1939] T 9257ca46dbb945bbbf4a37efd1512ec3 P 1cc8d7149e8a4a3d898b3043f7a8ba80: Deleting consensus metadata
I20260810 18:41:33.068223 8264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a8255f74b2a483fae3bfd9113829865. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:33.070976 8544 tablet_bootstrap.cc:492] T 8a8255f74b2a483fae3bfd9113829865 P 1cc8d7149e8a4a3d898b3043f7a8ba80: Bootstrap starting.
I20260810 18:41:33.074532 8544 tablet_bootstrap.cc:654] T 8a8255f74b2a483fae3bfd9113829865 P 1cc8d7149e8a4a3d898b3043f7a8ba80: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:33.075515 8547 tablet_bootstrap.cc:492] T 8a8255f74b2a483fae3bfd9113829865 P 15095ef6292f44f4a66a82e97497bb20: Bootstrap starting.
I20260810 18:41:33.078222 8547 tablet_bootstrap.cc:654] T 8a8255f74b2a483fae3bfd9113829865 P 15095ef6292f44f4a66a82e97497bb20: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:33.078392 8544 tablet_bootstrap.cc:492] T 8a8255f74b2a483fae3bfd9113829865 P 1cc8d7149e8a4a3d898b3043f7a8ba80: No bootstrap required, opened a new log
I20260810 18:41:33.078730 8544 ts_tablet_manager.cc:1397] T 8a8255f74b2a483fae3bfd9113829865 P 1cc8d7149e8a4a3d898b3043f7a8ba80: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260810 18:41:33.080240 8544 raft_consensus.cc:348] T 8a8255f74b2a483fae3bfd9113829865 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:33.080621 8544 raft_consensus.cc:374] T 8a8255f74b2a483fae3bfd9113829865 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:33.080761 8544 raft_consensus.cc:729] T 8a8255f74b2a483fae3bfd9113829865 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc8d7149e8a4a3d898b3043f7a8ba80, State: Initialized, Role: FOLLOWER
I20260810 18:41:33.081164 8544 consensus_queue.cc:260] T 8a8255f74b2a483fae3bfd9113829865 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:33.081928 8547 tablet_bootstrap.cc:492] T 8a8255f74b2a483fae3bfd9113829865 P 15095ef6292f44f4a66a82e97497bb20: No bootstrap required, opened a new log
I20260810 18:41:33.082172 8544 ts_tablet_manager.cc:1428] T 8a8255f74b2a483fae3bfd9113829865 P 1cc8d7149e8a4a3d898b3043f7a8ba80: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 18:41:33.082211 8547 ts_tablet_manager.cc:1397] T 8a8255f74b2a483fae3bfd9113829865 P 15095ef6292f44f4a66a82e97497bb20: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260810 18:41:33.083408 8547 raft_consensus.cc:348] T 8a8255f74b2a483fae3bfd9113829865 P 15095ef6292f44f4a66a82e97497bb20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:33.083781 8547 raft_consensus.cc:374] T 8a8255f74b2a483fae3bfd9113829865 P 15095ef6292f44f4a66a82e97497bb20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:33.083923 8547 raft_consensus.cc:729] T 8a8255f74b2a483fae3bfd9113829865 P 15095ef6292f44f4a66a82e97497bb20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15095ef6292f44f4a66a82e97497bb20, State: Initialized, Role: FOLLOWER
I20260810 18:41:33.084323 8547 consensus_queue.cc:260] T 8a8255f74b2a483fae3bfd9113829865 P 15095ef6292f44f4a66a82e97497bb20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:33.085078 8547 ts_tablet_manager.cc:1428] T 8a8255f74b2a483fae3bfd9113829865 P 15095ef6292f44f4a66a82e97497bb20: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 18:41:33.148203 7638 catalog_manager.cc:4841] TS 1cc8d7149e8a4a3d898b3043f7a8ba80 (127.4.228.194:43957): tablet 9257ca46dbb945bbbf4a37efd1512ec3 (table test-table [id=74de47ff46df4ac082cefe4cd2acfaa0]) successfully deleted
I20260810 18:41:33.162914 8528 ts_tablet_manager.cc:1918] T 9257ca46dbb945bbbf4a37efd1512ec3 P 15095ef6292f44f4a66a82e97497bb20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:41:33.162894 8532 ts_tablet_manager.cc:1918] T 9257ca46dbb945bbbf4a37efd1512ec3 P 06aed71215d44a608962bc2920f6f5dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:41:33.163137 8532 log.cc:1196] T 9257ca46dbb945bbbf4a37efd1512ec3 P 06aed71215d44a608962bc2920f6f5dc: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-2/wal/wals/9257ca46dbb945bbbf4a37efd1512ec3
I20260810 18:41:33.163160 8528 log.cc:1196] T 9257ca46dbb945bbbf4a37efd1512ec3 P 15095ef6292f44f4a66a82e97497bb20: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-0/wal/wals/9257ca46dbb945bbbf4a37efd1512ec3
I20260810 18:41:33.163635 8532 ts_tablet_manager.cc:1939] T 9257ca46dbb945bbbf4a37efd1512ec3 P 06aed71215d44a608962bc2920f6f5dc: Deleting consensus metadata
I20260810 18:41:33.163750 8528 ts_tablet_manager.cc:1939] T 9257ca46dbb945bbbf4a37efd1512ec3 P 15095ef6292f44f4a66a82e97497bb20: Deleting consensus metadata
I20260810 18:41:33.165135 7639 catalog_manager.cc:4841] TS 06aed71215d44a608962bc2920f6f5dc (127.4.228.195:45901): tablet 9257ca46dbb945bbbf4a37efd1512ec3 (table test-table [id=74de47ff46df4ac082cefe4cd2acfaa0]) successfully deleted
I20260810 18:41:33.165512 7635 catalog_manager.cc:4841] TS 15095ef6292f44f4a66a82e97497bb20 (127.4.228.193:43765): tablet 9257ca46dbb945bbbf4a37efd1512ec3 (table test-table [id=74de47ff46df4ac082cefe4cd2acfaa0]) successfully deleted
I20260810 18:41:33.168372 8580 tablet_bootstrap.cc:492] T 8a8255f74b2a483fae3bfd9113829865 P 06aed71215d44a608962bc2920f6f5dc: Bootstrap starting.
I20260810 18:41:33.170665 8580 tablet_bootstrap.cc:654] T 8a8255f74b2a483fae3bfd9113829865 P 06aed71215d44a608962bc2920f6f5dc: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:33.173674 8580 tablet_bootstrap.cc:492] T 8a8255f74b2a483fae3bfd9113829865 P 06aed71215d44a608962bc2920f6f5dc: No bootstrap required, opened a new log
I20260810 18:41:33.173916 8580 ts_tablet_manager.cc:1397] T 8a8255f74b2a483fae3bfd9113829865 P 06aed71215d44a608962bc2920f6f5dc: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260810 18:41:33.175096 8580 raft_consensus.cc:348] T 8a8255f74b2a483fae3bfd9113829865 P 06aed71215d44a608962bc2920f6f5dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:33.175413 8580 raft_consensus.cc:374] T 8a8255f74b2a483fae3bfd9113829865 P 06aed71215d44a608962bc2920f6f5dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:33.175591 8580 raft_consensus.cc:729] T 8a8255f74b2a483fae3bfd9113829865 P 06aed71215d44a608962bc2920f6f5dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06aed71215d44a608962bc2920f6f5dc, State: Initialized, Role: FOLLOWER
I20260810 18:41:33.175951 8580 consensus_queue.cc:260] T 8a8255f74b2a483fae3bfd9113829865 P 06aed71215d44a608962bc2920f6f5dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:33.176716 8580 ts_tablet_manager.cc:1428] T 8a8255f74b2a483fae3bfd9113829865 P 06aed71215d44a608962bc2920f6f5dc: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260810 18:41:33.513592 8495 raft_consensus.cc:482] T 8a8255f74b2a483fae3bfd9113829865 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:41:33.513844 8495 raft_consensus.cc:504] T 8a8255f74b2a483fae3bfd9113829865 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:33.514760 8495 leader_election.cc:288] T 8a8255f74b2a483fae3bfd9113829865 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15095ef6292f44f4a66a82e97497bb20 (127.4.228.193:43765), 06aed71215d44a608962bc2920f6f5dc (127.4.228.195:45901)
I20260810 18:41:33.515161 8520 raft_consensus.cc:482] T 8a8255f74b2a483fae3bfd9113829865 P 06aed71215d44a608962bc2920f6f5dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:41:33.515623 8293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a8255f74b2a483fae3bfd9113829865" candidate_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06aed71215d44a608962bc2920f6f5dc" is_pre_election: true
I20260810 18:41:33.515575 8520 raft_consensus.cc:504] T 8a8255f74b2a483fae3bfd9113829865 P 06aed71215d44a608962bc2920f6f5dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:33.516232 8293 raft_consensus.cc:2454] T 8a8255f74b2a483fae3bfd9113829865 P 06aed71215d44a608962bc2920f6f5dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cc8d7149e8a4a3d898b3043f7a8ba80 in term 0.
I20260810 18:41:33.516136 7886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a8255f74b2a483fae3bfd9113829865" candidate_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15095ef6292f44f4a66a82e97497bb20" is_pre_election: true
I20260810 18:41:33.516485 7886 raft_consensus.cc:2454] T 8a8255f74b2a483fae3bfd9113829865 P 15095ef6292f44f4a66a82e97497bb20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cc8d7149e8a4a3d898b3043f7a8ba80 in term 0.
I20260810 18:41:33.516825 8520 leader_election.cc:288] T 8a8255f74b2a483fae3bfd9113829865 P 06aed71215d44a608962bc2920f6f5dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15095ef6292f44f4a66a82e97497bb20 (127.4.228.193:43765), 1cc8d7149e8a4a3d898b3043f7a8ba80 (127.4.228.194:43957)
I20260810 18:41:33.516983 7994 leader_election.cc:302] T 8a8255f74b2a483fae3bfd9113829865 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06aed71215d44a608962bc2920f6f5dc, 1cc8d7149e8a4a3d898b3043f7a8ba80; no voters:
I20260810 18:41:33.517344 7886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a8255f74b2a483fae3bfd9113829865" candidate_uuid: "06aed71215d44a608962bc2920f6f5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15095ef6292f44f4a66a82e97497bb20" is_pre_election: true
I20260810 18:41:33.517346 8092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a8255f74b2a483fae3bfd9113829865" candidate_uuid: "06aed71215d44a608962bc2920f6f5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" is_pre_election: true
I20260810 18:41:33.517483 8495 raft_consensus.cc:2781] T 8a8255f74b2a483fae3bfd9113829865 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:41:33.517678 7886 raft_consensus.cc:2454] T 8a8255f74b2a483fae3bfd9113829865 P 15095ef6292f44f4a66a82e97497bb20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06aed71215d44a608962bc2920f6f5dc in term 0.
I20260810 18:41:33.517683 8092 raft_consensus.cc:2454] T 8a8255f74b2a483fae3bfd9113829865 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06aed71215d44a608962bc2920f6f5dc in term 0.
I20260810 18:41:33.517988 8495 raft_consensus.cc:482] T 8a8255f74b2a483fae3bfd9113829865 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:41:33.518224 8495 raft_consensus.cc:3037] T 8a8255f74b2a483fae3bfd9113829865 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:33.518324 8190 leader_election.cc:302] T 8a8255f74b2a483fae3bfd9113829865 P 06aed71215d44a608962bc2920f6f5dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06aed71215d44a608962bc2920f6f5dc, 15095ef6292f44f4a66a82e97497bb20; no voters:
I20260810 18:41:33.518739 8520 raft_consensus.cc:2781] T 8a8255f74b2a483fae3bfd9113829865 P 06aed71215d44a608962bc2920f6f5dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:41:33.518970 8520 raft_consensus.cc:482] T 8a8255f74b2a483fae3bfd9113829865 P 06aed71215d44a608962bc2920f6f5dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:41:33.519122 8520 raft_consensus.cc:3037] T 8a8255f74b2a483fae3bfd9113829865 P 06aed71215d44a608962bc2920f6f5dc [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:33.567802 8520 raft_consensus.cc:504] T 8a8255f74b2a483fae3bfd9113829865 P 06aed71215d44a608962bc2920f6f5dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:33.567929 8495 raft_consensus.cc:504] T 8a8255f74b2a483fae3bfd9113829865 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:33.569280 8520 leader_election.cc:288] T 8a8255f74b2a483fae3bfd9113829865 P 06aed71215d44a608962bc2920f6f5dc [CANDIDATE]: Term 1 election: Requested vote from peers 15095ef6292f44f4a66a82e97497bb20 (127.4.228.193:43765), 1cc8d7149e8a4a3d898b3043f7a8ba80 (127.4.228.194:43957)
I20260810 18:41:33.569554 8495 leader_election.cc:288] T 8a8255f74b2a483fae3bfd9113829865 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [CANDIDATE]: Term 1 election: Requested vote from peers 15095ef6292f44f4a66a82e97497bb20 (127.4.228.193:43765), 06aed71215d44a608962bc2920f6f5dc (127.4.228.195:45901)
I20260810 18:41:33.569628 7885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a8255f74b2a483fae3bfd9113829865" candidate_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15095ef6292f44f4a66a82e97497bb20"
I20260810 18:41:33.569739 8293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a8255f74b2a483fae3bfd9113829865" candidate_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06aed71215d44a608962bc2920f6f5dc"
I20260810 18:41:33.569941 7885 raft_consensus.cc:3037] T 8a8255f74b2a483fae3bfd9113829865 P 15095ef6292f44f4a66a82e97497bb20 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:33.569968 7886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a8255f74b2a483fae3bfd9113829865" candidate_uuid: "06aed71215d44a608962bc2920f6f5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15095ef6292f44f4a66a82e97497bb20"
I20260810 18:41:33.570140 8293 raft_consensus.cc:2379] T 8a8255f74b2a483fae3bfd9113829865 P 06aed71215d44a608962bc2920f6f5dc [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1cc8d7149e8a4a3d898b3043f7a8ba80 in current term 1: Already voted for candidate 06aed71215d44a608962bc2920f6f5dc in this term.
I20260810 18:41:33.570633 8092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a8255f74b2a483fae3bfd9113829865" candidate_uuid: "06aed71215d44a608962bc2920f6f5dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80"
I20260810 18:41:33.570981 8092 raft_consensus.cc:2379] T 8a8255f74b2a483fae3bfd9113829865 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 06aed71215d44a608962bc2920f6f5dc in current term 1: Already voted for candidate 1cc8d7149e8a4a3d898b3043f7a8ba80 in this term.
I20260810 18:41:33.571581 8195 leader_election.cc:302] T 8a8255f74b2a483fae3bfd9113829865 P 06aed71215d44a608962bc2920f6f5dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 06aed71215d44a608962bc2920f6f5dc; no voters: 15095ef6292f44f4a66a82e97497bb20, 1cc8d7149e8a4a3d898b3043f7a8ba80
I20260810 18:41:33.571942 8520 raft_consensus.cc:2726] T 8a8255f74b2a483fae3bfd9113829865 P 06aed71215d44a608962bc2920f6f5dc [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260810 18:41:33.676914 7885 raft_consensus.cc:2454] T 8a8255f74b2a483fae3bfd9113829865 P 15095ef6292f44f4a66a82e97497bb20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cc8d7149e8a4a3d898b3043f7a8ba80 in term 1.
I20260810 18:41:33.677726 7991 leader_election.cc:302] T 8a8255f74b2a483fae3bfd9113829865 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 15095ef6292f44f4a66a82e97497bb20, 1cc8d7149e8a4a3d898b3043f7a8ba80; no voters: 06aed71215d44a608962bc2920f6f5dc
I20260810 18:41:33.678145 8495 raft_consensus.cc:2781] T 8a8255f74b2a483fae3bfd9113829865 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:41:33.678637 8495 raft_consensus.cc:686] T 8a8255f74b2a483fae3bfd9113829865 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 1 LEADER]: Becoming Leader. State: Replica: 1cc8d7149e8a4a3d898b3043f7a8ba80, State: Running, Role: LEADER
I20260810 18:41:33.679098 8495 consensus_queue.cc:237] T 8a8255f74b2a483fae3bfd9113829865 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } }
I20260810 18:41:33.682230 7654 catalog_manager.cc:5495] T 8a8255f74b2a483fae3bfd9113829865 P 1cc8d7149e8a4a3d898b3043f7a8ba80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1cc8d7149e8a4a3d898b3043f7a8ba80 (127.4.228.194). New cstate: current_term: 1 leader_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cc8d7149e8a4a3d898b3043f7a8ba80" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 } health_report { overall_health: UNKNOWN } } }
I20260810 18:41:33.816017 7885 raft_consensus.cc:1261] T 8a8255f74b2a483fae3bfd9113829865 P 15095ef6292f44f4a66a82e97497bb20 [term 1 FOLLOWER]: Refusing update from remote peer 1cc8d7149e8a4a3d898b3043f7a8ba80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:41:33.816022 8293 raft_consensus.cc:1261] T 8a8255f74b2a483fae3bfd9113829865 P 06aed71215d44a608962bc2920f6f5dc [term 1 FOLLOWER]: Refusing update from remote peer 1cc8d7149e8a4a3d898b3043f7a8ba80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:41:33.816771 8585 consensus_queue.cc:1035] T 8a8255f74b2a483fae3bfd9113829865 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15095ef6292f44f4a66a82e97497bb20" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:41:33.817186 8597 consensus_queue.cc:1035] T 8a8255f74b2a483fae3bfd9113829865 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06aed71215d44a608962bc2920f6f5dc" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 18:41:33.859102 8591 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 18:41:35.474030 7654 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:41:35.474408 7654 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:41:35.480475 7654 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a3ca18b5d0844726a51e2587276c30d9: Sending DeleteTablet for 3 replicas of tablet 8a8255f74b2a483fae3bfd9113829865
I20260810 18:41:35.481869 7855 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a8255f74b2a483fae3bfd9113829865 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:41:35 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:58080
I20260810 18:41:35.481976 8264 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a8255f74b2a483fae3bfd9113829865 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:41:35 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:55444
I20260810 18:41:35.482077 8059 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a8255f74b2a483fae3bfd9113829865 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:41:35 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:50780
I20260810 18:41:35.482985 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 7758
I20260810 18:41:35.495070 8645 tablet_replica.cc:330] T 8a8255f74b2a483fae3bfd9113829865 P 06aed71215d44a608962bc2920f6f5dc: stopping tablet replica
I20260810 18:41:35.495615 8645 raft_consensus.cc:2229] T 8a8255f74b2a483fae3bfd9113829865 P 06aed71215d44a608962bc2920f6f5dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:41:35.496300 8645 raft_consensus.cc:2258] T 8a8255f74b2a483fae3bfd9113829865 P 06aed71215d44a608962bc2920f6f5dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:41:35.497717 8644 tablet_replica.cc:330] T 8a8255f74b2a483fae3bfd9113829865 P 15095ef6292f44f4a66a82e97497bb20: stopping tablet replica
I20260810 18:41:35.497991 8644 raft_consensus.cc:2229] T 8a8255f74b2a483fae3bfd9113829865 P 15095ef6292f44f4a66a82e97497bb20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:41:35.498317 8644 raft_consensus.cc:2258] T 8a8255f74b2a483fae3bfd9113829865 P 15095ef6292f44f4a66a82e97497bb20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:41:35.498608 8645 ts_tablet_manager.cc:1905] T 8a8255f74b2a483fae3bfd9113829865 P 06aed71215d44a608962bc2920f6f5dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:41:35.499593 8647 tablet_replica.cc:330] T 8a8255f74b2a483fae3bfd9113829865 P 1cc8d7149e8a4a3d898b3043f7a8ba80: stopping tablet replica
I20260810 18:41:35.499963 8647 raft_consensus.cc:2229] T 8a8255f74b2a483fae3bfd9113829865 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:41:35.500124 8644 ts_tablet_manager.cc:1905] T 8a8255f74b2a483fae3bfd9113829865 P 15095ef6292f44f4a66a82e97497bb20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:41:35.500569 8647 raft_consensus.cc:2258] T 8a8255f74b2a483fae3bfd9113829865 P 1cc8d7149e8a4a3d898b3043f7a8ba80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:41:35.501971 8647 ts_tablet_manager.cc:1905] T 8a8255f74b2a483fae3bfd9113829865 P 1cc8d7149e8a4a3d898b3043f7a8ba80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:41:35.508934 8644 ts_tablet_manager.cc:1918] T 8a8255f74b2a483fae3bfd9113829865 P 15095ef6292f44f4a66a82e97497bb20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 18:41:35.509176 8644 log.cc:1196] T 8a8255f74b2a483fae3bfd9113829865 P 15095ef6292f44f4a66a82e97497bb20: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-0/wal/wals/8a8255f74b2a483fae3bfd9113829865
I20260810 18:41:35.509936 8644 ts_tablet_manager.cc:1939] T 8a8255f74b2a483fae3bfd9113829865 P 15095ef6292f44f4a66a82e97497bb20: Deleting consensus metadata
I20260810 18:41:35.510165 8645 ts_tablet_manager.cc:1918] T 8a8255f74b2a483fae3bfd9113829865 P 06aed71215d44a608962bc2920f6f5dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 18:41:35.510437 8645 log.cc:1196] T 8a8255f74b2a483fae3bfd9113829865 P 06aed71215d44a608962bc2920f6f5dc: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-2/wal/wals/8a8255f74b2a483fae3bfd9113829865
I20260810 18:41:35.511219 8645 ts_tablet_manager.cc:1939] T 8a8255f74b2a483fae3bfd9113829865 P 06aed71215d44a608962bc2920f6f5dc: Deleting consensus metadata
I20260810 18:41:35.511495 7635 catalog_manager.cc:4841] TS 15095ef6292f44f4a66a82e97497bb20 (127.4.228.193:43765): tablet 8a8255f74b2a483fae3bfd9113829865 (table test-table [id=fc1d3f21a2f1402f9b83ba469a5ffa2e]) successfully deleted
I20260810 18:41:35.512127 8647 ts_tablet_manager.cc:1918] T 8a8255f74b2a483fae3bfd9113829865 P 1cc8d7149e8a4a3d898b3043f7a8ba80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 18:41:35.512413 8647 log.cc:1196] T 8a8255f74b2a483fae3bfd9113829865 P 1cc8d7149e8a4a3d898b3043f7a8ba80: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387267874959-5011-0/minicluster-data/ts-1/wal/wals/8a8255f74b2a483fae3bfd9113829865
I20260810 18:41:35.512678 7639 catalog_manager.cc:4841] TS 06aed71215d44a608962bc2920f6f5dc (127.4.228.195:45901): tablet 8a8255f74b2a483fae3bfd9113829865 (table test-table [id=fc1d3f21a2f1402f9b83ba469a5ffa2e]) successfully deleted
I20260810 18:41:35.513124 8647 ts_tablet_manager.cc:1939] T 8a8255f74b2a483fae3bfd9113829865 P 1cc8d7149e8a4a3d898b3043f7a8ba80: Deleting consensus metadata
I20260810 18:41:35.514103 7638 catalog_manager.cc:4841] TS 1cc8d7149e8a4a3d898b3043f7a8ba80 (127.4.228.194:43957): tablet 8a8255f74b2a483fae3bfd9113829865 (table test-table [id=fc1d3f21a2f1402f9b83ba469a5ffa2e]) successfully deleted
Aug 10 18:41:35 dist-test-slave-7kzw krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387292, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.228.193@KRBTEST.COM
I20260810 18:41:35.526418 8650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:35.485075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55278 (local address 127.4.228.193:43765)
0810 18:41:35.493245 (+ 8170us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:35.493255 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:35.494286 (+ 1031us) server_negotiation.cc:407] Connection header received
0810 18:41:35.494809 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:35.494821 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:35.494961 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:35.498882 (+ 3921us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:35.499162 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:35.501171 (+ 2009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:35.503715 (+ 2544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:35.504701 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:35.504880 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:35.516503 (+ 11623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:35.516537 (+ 34us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:35.516557 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:35.516605 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:35.521724 (+ 5119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:35.522920 (+ 1196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:35.522930 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:35.522933 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:35.523188 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:35.523895 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:35.523903 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:35.523907 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:35.524748 (+ 841us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:35.525202 (+ 454us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:35.525565 (+ 363us) server_negotiation.cc:299] Negotiation successful
0810 18:41:35.525864 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7892,"thread_start_us":7596,"threads_started":1}
W20260810 18:41:35.527375 7957 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55278
W20260810 18:41:35.528290 5011 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 7758: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:41:35.528664 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 7758
I20260810 18:41:35.563881 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 7971
Aug 10 18:41:35 dist-test-slave-7kzw krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387292, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.228.194@KRBTEST.COM
I20260810 18:41:35.583956 8652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:35.565292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55788 (local address 127.4.228.194:43957)
0810 18:41:35.566192 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:35.566197 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:35.566216 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:41:35.566275 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:35.566279 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:35.566351 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:35.566454 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:35.568144 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:35.568936 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:35.570394 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:35.570954 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:35.571075 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:35.576220 (+ 5145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:35.576241 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:35.576263 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:35.576311 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:35.580017 (+ 3706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:35.580952 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:35.580962 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:35.580964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:35.581156 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:35.581736 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:35.581749 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:35.581751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:35.582229 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:35.582582 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:35.583175 (+ 593us) server_negotiation.cc:299] Negotiation successful
0810 18:41:35.583477 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":766,"thread_start_us":557,"threads_started":1}
W20260810 18:41:35.584826 8152 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55788
W20260810 18:41:35.585801 5011 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 7971: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:41:35.586035 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 7971
I20260810 18:41:35.632925 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 8167
Aug 10 18:41:35 dist-test-slave-7kzw krb5kdc[7591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387292, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.228.195@KRBTEST.COM
I20260810 18:41:35.653324 8653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:35.633935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52476 (local address 127.4.228.195:45901)
0810 18:41:35.634793 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:35.634798 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:35.634816 (+ 18us) server_negotiation.cc:407] Connection header received
0810 18:41:35.634878 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:35.634881 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:35.634956 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:35.635056 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:35.636929 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:35.637773 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:35.639357 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:35.640075 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:35.640204 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:35.645889 (+ 5685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:35.645909 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:35.645929 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:35.645977 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:35.649508 (+ 3531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:35.650684 (+ 1176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:35.650715 (+ 31us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:35.650718 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:35.650936 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:35.651448 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:35.651453 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:35.651455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:35.651945 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:35.652213 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:35.652687 (+ 474us) server_negotiation.cc:299] Negotiation successful
0810 18:41:35.652898 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":701,"thread_start_us":379,"threads_started":1}
W20260810 18:41:35.654314 8364 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52476
W20260810 18:41:35.655289 5011 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 8167: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:41:35.655462 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 8167
I20260810 18:41:35.698220 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 7610
I20260810 18:41:35.699690 7715 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:41:35.826807 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 7610
2026-08-10T18:41:35Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (10114 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:41:35 dist-test-slave-7kzw krb5kdc[8661](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:41:35 dist-test-slave-7kzw krb5kdc[8661](info): set up 2 sockets
Aug 10 18:41:35 dist-test-slave-7kzw krb5kdc[8661](info): commencing operation
krb5kdc: starting...
W20260810 18:41:37.222971 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.335s user 0.000s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:41:37 dist-test-slave-7kzw krb5kdc[8661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:41:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:41:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.228.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
I20260810 18:41:37.487442 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:34529
--webserver_interface=127.4.228.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:41567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:34529
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:41:37.859577 8680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:41:37.859918 8680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:41:37.860091 8680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:41:37.867820 8680 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:41:37.867959 8680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:41:37.868045 8680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:41:37.868135 8680 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:41:37.868201 8680 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:41:37.878944 8680 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:41567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:34529
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:34529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:41:37.880466 8680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:41:37.882009 8680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 18:41:37.896751 8680 server_base.cc:1034] running on GCE node
W20260810 18:41:37.896689 8689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:37.896956 8691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:37.896638 8686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:37.898228 8680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:41:37.900028 8680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:41:37.901309 8680 hybrid_clock.cc:648] HybridClock initialized: now 1786387297901241 us; error 57 us; skew 500 ppm
I20260810 18:41:37.901811 8680 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:41:37 dist-test-slave-7kzw krb5kdc[8661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387297, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:41:37.905890 8680 init.cc:377] Logged in from keytab as kudu/127.4.228.254@KRBTEST.COM (short username kudu)
I20260810 18:41:37.907888 8680 webserver.cc:460] Webserver started at http://127.4.228.254:34365/ using document root <none> and password file <none>
I20260810 18:41:37.908782 8680 fs_manager.cc:359] Metadata directory not provided
I20260810 18:41:37.908943 8680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:41:37.909349 8680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:41:37.913017 8680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "e591cfa72de24a12af3693b2f67b7e92"
format_stamp: "Formatted at 2026-08-10 18:41:37 on dist-test-slave-7kzw"
server_key: "ffb8ccda5c0d3ab5b378fd849353eb14"
server_key_iv: "7a450c0dccfcb0e67cda8d65febb4bf0"
server_key_version: "encryptionkey@0"
I20260810 18:41:37.914050 8680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "e591cfa72de24a12af3693b2f67b7e92"
format_stamp: "Formatted at 2026-08-10 18:41:37 on dist-test-slave-7kzw"
server_key: "ffb8ccda5c0d3ab5b378fd849353eb14"
server_key_iv: "7a450c0dccfcb0e67cda8d65febb4bf0"
server_key_version: "encryptionkey@0"
I20260810 18:41:37.920461 8680 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260810 18:41:37.924923 8700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:37.926833 8680 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 18:41:37.927121 8680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "e591cfa72de24a12af3693b2f67b7e92"
format_stamp: "Formatted at 2026-08-10 18:41:37 on dist-test-slave-7kzw"
server_key: "ffb8ccda5c0d3ab5b378fd849353eb14"
server_key_iv: "7a450c0dccfcb0e67cda8d65febb4bf0"
server_key_version: "encryptionkey@0"
I20260810 18:41:37.927426 8680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:41:37.952845 8680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:41:37.957882 8680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:41:37.958238 8680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:41:37.979221 8680 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:34529
I20260810 18:41:37.979244 8781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:34529 every 8 connection(s)
I20260810 18:41:37.980854 8680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:41:37.985240 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 8680
I20260810 18:41:37.985541 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:41:37.985559 8782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:37.986409 5011 external_mini_cluster.cc:1428] Setting key d592e6f07627109f9952d7aeb979c13e
I20260810 18:41:38.000579 8782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e591cfa72de24a12af3693b2f67b7e92: Bootstrap starting.
Aug 10 18:41:37 dist-test-slave-7kzw krb5kdc[8661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:38.005577 8782 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e591cfa72de24a12af3693b2f67b7e92: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:38.007126 8782 log.cc:824] T 00000000000000000000000000000000 P e591cfa72de24a12af3693b2f67b7e92: Log is configured to *not* fsync() on all Append() calls
I20260810 18:41:38.010737 8782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e591cfa72de24a12af3693b2f67b7e92: No bootstrap required, opened a new log
I20260810 18:41:38.011953 8789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:37.989008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41054 (local address 127.4.228.254:34529)
0810 18:41:37.990042 (+ 1034us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:37.990056 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:37.990107 (+ 51us) server_negotiation.cc:407] Connection header received
0810 18:41:37.991839 (+ 1732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:37.991864 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:37.992313 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:37.992788 (+ 475us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:37.994451 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:37.995837 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:37.997313 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:37.998121 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:37.998346 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:38.003099 (+ 4753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:38.003109 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:38.003131 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:38.003189 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:38.007348 (+ 4159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:38.008242 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:38.008250 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:38.008257 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:38.008451 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:38.008885 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:38.008891 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:38.008896 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:38.009468 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:38.009816 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:38.010287 (+ 471us) server_negotiation.cc:299] Negotiation successful
0810 18:41:38.010804 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":794,"thread_start_us":473,"threads_started":1}
I20260810 18:41:38.020574 8782 raft_consensus.cc:348] T 00000000000000000000000000000000 P e591cfa72de24a12af3693b2f67b7e92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e591cfa72de24a12af3693b2f67b7e92" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 34529 } }
I20260810 18:41:38.021060 8782 raft_consensus.cc:374] T 00000000000000000000000000000000 P e591cfa72de24a12af3693b2f67b7e92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:38.021212 8782 raft_consensus.cc:729] T 00000000000000000000000000000000 P e591cfa72de24a12af3693b2f67b7e92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e591cfa72de24a12af3693b2f67b7e92, State: Initialized, Role: FOLLOWER
I20260810 18:41:38.021899 8782 consensus_queue.cc:260] T 00000000000000000000000000000000 P e591cfa72de24a12af3693b2f67b7e92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e591cfa72de24a12af3693b2f67b7e92" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 34529 } }
I20260810 18:41:38.022156 8782 raft_consensus.cc:388] T 00000000000000000000000000000000 P e591cfa72de24a12af3693b2f67b7e92 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:41:38.022327 8782 raft_consensus.cc:482] T 00000000000000000000000000000000 P e591cfa72de24a12af3693b2f67b7e92 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:41:38.022553 8782 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e591cfa72de24a12af3693b2f67b7e92 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:38.025036 8782 raft_consensus.cc:504] T 00000000000000000000000000000000 P e591cfa72de24a12af3693b2f67b7e92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e591cfa72de24a12af3693b2f67b7e92" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 34529 } }
I20260810 18:41:38.025691 8782 leader_election.cc:302] T 00000000000000000000000000000000 P e591cfa72de24a12af3693b2f67b7e92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e591cfa72de24a12af3693b2f67b7e92; no voters:
I20260810 18:41:38.026551 8782 leader_election.cc:288] T 00000000000000000000000000000000 P e591cfa72de24a12af3693b2f67b7e92 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:41:38.026743 8791 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e591cfa72de24a12af3693b2f67b7e92 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:41:38.027613 8791 raft_consensus.cc:686] T 00000000000000000000000000000000 P e591cfa72de24a12af3693b2f67b7e92 [term 1 LEADER]: Becoming Leader. State: Replica: e591cfa72de24a12af3693b2f67b7e92, State: Running, Role: LEADER
I20260810 18:41:38.028247 8791 consensus_queue.cc:237] T 00000000000000000000000000000000 P e591cfa72de24a12af3693b2f67b7e92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e591cfa72de24a12af3693b2f67b7e92" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 34529 } }
I20260810 18:41:38.029088 8782 sys_catalog.cc:564] T 00000000000000000000000000000000 P e591cfa72de24a12af3693b2f67b7e92 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:41:38.031874 8793 sys_catalog.cc:455] T 00000000000000000000000000000000 P e591cfa72de24a12af3693b2f67b7e92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e591cfa72de24a12af3693b2f67b7e92. Latest consensus state: current_term: 1 leader_uuid: "e591cfa72de24a12af3693b2f67b7e92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e591cfa72de24a12af3693b2f67b7e92" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 34529 } } }
I20260810 18:41:38.031762 8792 sys_catalog.cc:455] T 00000000000000000000000000000000 P e591cfa72de24a12af3693b2f67b7e92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e591cfa72de24a12af3693b2f67b7e92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e591cfa72de24a12af3693b2f67b7e92" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 34529 } } }
I20260810 18:41:38.032341 8793 sys_catalog.cc:458] T 00000000000000000000000000000000 P e591cfa72de24a12af3693b2f67b7e92 [sys.catalog]: This master's current role is: LEADER
I20260810 18:41:38.032392 8792 sys_catalog.cc:458] T 00000000000000000000000000000000 P e591cfa72de24a12af3693b2f67b7e92 [sys.catalog]: This master's current role is: LEADER
I20260810 18:41:38.039315 8803 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:41:38.045086 8803 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:41:38.054970 8803 catalog_manager.cc:1347] Generated new cluster ID: 7e8a2b6e15124571894b8a8cadaed31c
I20260810 18:41:38.055114 8803 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:41:38.074455 8803 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:41:38.076576 8803 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:41:38.088999 8803 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e591cfa72de24a12af3693b2f67b7e92: Generated new TSK 0
I20260810 18:41:38.089948 8803 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.228.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab.
Entry for principal kudu/127.4.228.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab.
I20260810 18:41:38.163426 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.193:0
--local_ip_for_outbound_sockets=127.4.228.193
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:34529
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:41567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:41:38.543408 8819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:41:38.543787 8819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:41:38.543949 8819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:41:38.551512 8819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:41:38.551679 8819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:41:38.551860 8819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.193
I20260810 18:41:38.562193 8819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:41567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:34529
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.193
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:41:38.563725 8819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:41:38.565147 8819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:41:38.580008 8827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:38.580060 8826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:38.579962 8833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:38.581116 8819 server_base.cc:1034] running on GCE node
I20260810 18:41:38.581857 8819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:41:38.583001 8819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:41:38.584228 8819 hybrid_clock.cc:648] HybridClock initialized: now 1786387298584175 us; error 40 us; skew 500 ppm
I20260810 18:41:38.584602 8819 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:41:38 dist-test-slave-7kzw krb5kdc[8661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387298, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:41:38.588253 8819 init.cc:377] Logged in from keytab as kudu/127.4.228.193@KRBTEST.COM (short username kudu)
I20260810 18:41:38.590071 8819 webserver.cc:460] Webserver started at http://127.4.228.193:45055/ using document root <none> and password file <none>
I20260810 18:41:38.590936 8819 fs_manager.cc:359] Metadata directory not provided
I20260810 18:41:38.591053 8819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:41:38.591434 8819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:41:38.594727 8819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-0/data/instance:
uuid: "e8225bf44d7840b0b7e65db5f9dca10c"
format_stamp: "Formatted at 2026-08-10 18:41:38 on dist-test-slave-7kzw"
server_key: "a4f599c2409ee65a780462150aa7d7ec"
server_key_iv: "2a256b00d5683e4bc110a202bb1caa9b"
server_key_version: "encryptionkey@0"
I20260810 18:41:38.595634 8819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance:
uuid: "e8225bf44d7840b0b7e65db5f9dca10c"
format_stamp: "Formatted at 2026-08-10 18:41:38 on dist-test-slave-7kzw"
server_key: "a4f599c2409ee65a780462150aa7d7ec"
server_key_iv: "2a256b00d5683e4bc110a202bb1caa9b"
server_key_version: "encryptionkey@0"
I20260810 18:41:38.602411 8819 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260810 18:41:38.606562 8844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:38.608269 8819 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:41:38.608500 8819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-0/wal
uuid: "e8225bf44d7840b0b7e65db5f9dca10c"
format_stamp: "Formatted at 2026-08-10 18:41:38 on dist-test-slave-7kzw"
server_key: "a4f599c2409ee65a780462150aa7d7ec"
server_key_iv: "2a256b00d5683e4bc110a202bb1caa9b"
server_key_version: "encryptionkey@0"
I20260810 18:41:38.608709 8819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:41:38.622794 8819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:41:38.627737 8819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:41:38.628063 8819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:41:38.629230 8819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:41:38.630964 8819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:41:38.631045 8819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:38.631168 8819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:41:38.631229 8819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:38.669459 8819 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.193:35569
I20260810 18:41:38.669488 9028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.193:35569 every 8 connection(s)
I20260810 18:41:38.670957 8819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
I20260810 18:41:38.671636 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 8819
I20260810 18:41:38.671898 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance
I20260810 18:41:38.672824 5011 external_mini_cluster.cc:1428] Setting key 8edfb3e86ab4cc70522e483f208dfdc6
Aug 10 18:41:38 dist-test-slave-7kzw krb5kdc[8661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387298, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:38.697723 9032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:38.674240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:46767 (local address 127.4.228.254:34529)
0810 18:41:38.674781 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:38.674789 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:38.676120 (+ 1331us) server_negotiation.cc:407] Connection header received
0810 18:41:38.679182 (+ 3062us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:38.679187 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:38.679275 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:38.679387 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:38.681715 (+ 2328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:38.682589 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:38.684790 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:38.685559 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:38.685746 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:38.690004 (+ 4258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:38.690010 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:38.690014 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:38.690056 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:38.693257 (+ 3201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:38.694781 (+ 1524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:38.694786 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:38.694789 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:38.694936 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:38.695570 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:38.695575 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:38.695577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:38.695914 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:38.696032 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:38.697228 (+ 1196us) server_negotiation.cc:299] Negotiation successful
0810 18:41:38.697439 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":296,"threads_started":1}
WARNING: no policy specified for kudu/127.4.228.194@KRBTEST.COM; defaulting to no policy
I20260810 18:41:38.699420 9033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:38.675003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:34529 (local address 127.4.228.193:46767)
0810 18:41:38.675920 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:38.675979 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:38.678833 (+ 2854us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:38.679687 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:38.679702 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:38.680590 (+ 888us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:38.681479 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:38.681510 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:38.682757 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:38.682765 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:38.684605 (+ 1840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:38.684626 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:38.685812 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:38.685818 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:38.686020 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:38.686876 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:41:38.686901 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:38.689685 (+ 2784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:38.693468 (+ 3783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:38.693484 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:38.693514 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:38.694362 (+ 848us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:38.695085 (+ 723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:38.695094 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:38.695100 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:38.695425 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:38.696095 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:38.696108 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:38.696688 (+ 580us) client_negotiation.cc:769] Sending connection context
0810 18:41:38.697065 (+ 377us) client_negotiation.cc:240] Negotiation successful
0810 18:41:38.697743 (+ 678us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":323,"threads_started":1}
I20260810 18:41:38.701676 9030 heartbeater.cc:347] Connected to a master server at 127.4.228.254:34529
I20260810 18:41:38.702009 9030 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.194@KRBTEST.COM" created.
I20260810 18:41:38.702955 9030 heartbeater.cc:511] Master 127.4.228.254:34529 requested a full tablet report, sending...
I20260810 18:41:38.705466 8728 ts_manager.cc:194] Registered new tserver with Master: e8225bf44d7840b0b7e65db5f9dca10c (127.4.228.193:35569)
I20260810 18:41:38.707120 8728 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.193@KRBTEST.COM'} at 127.4.228.193:46767
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab.
Entry for principal kudu/127.4.228.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab.
I20260810 18:41:38.735011 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.194:0
--local_ip_for_outbound_sockets=127.4.228.194
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:34529
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:41567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:41:39.118927 9036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:41:39.119263 9036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:41:39.119432 9036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:41:39.127100 9036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:41:39.127236 9036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:41:39.127429 9036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.194
I20260810 18:41:39.137990 9036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:41567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:34529
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.194
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:41:39.139596 9036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:41:39.141155 9036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:41:39.156833 9054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:39.156822 9050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:39.156828 9049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:39.158097 9036 server_base.cc:1034] running on GCE node
I20260810 18:41:39.158951 9036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:41:39.160089 9036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:41:39.161501 9036 hybrid_clock.cc:648] HybridClock initialized: now 1786387299161394 us; error 112 us; skew 500 ppm
I20260810 18:41:39.161994 9036 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:41:39 dist-test-slave-7kzw krb5kdc[8661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387299, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:41:39.165802 9036 init.cc:377] Logged in from keytab as kudu/127.4.228.194@KRBTEST.COM (short username kudu)
I20260810 18:41:39.167500 9036 webserver.cc:460] Webserver started at http://127.4.228.194:33515/ using document root <none> and password file <none>
I20260810 18:41:39.168253 9036 fs_manager.cc:359] Metadata directory not provided
I20260810 18:41:39.168362 9036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:41:39.168650 9036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:41:39.171948 9036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-1/data/instance:
uuid: "528ecdf5ef8c4a448ec7fb8dfb2f5e40"
format_stamp: "Formatted at 2026-08-10 18:41:39 on dist-test-slave-7kzw"
server_key: "ae510ba51bc474f5ff52d76717e29ba6"
server_key_iv: "b32f02684f4ad3f15d1c22407deb9c59"
server_key_version: "encryptionkey@0"
I20260810 18:41:39.172796 9036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance:
uuid: "528ecdf5ef8c4a448ec7fb8dfb2f5e40"
format_stamp: "Formatted at 2026-08-10 18:41:39 on dist-test-slave-7kzw"
server_key: "ae510ba51bc474f5ff52d76717e29ba6"
server_key_iv: "b32f02684f4ad3f15d1c22407deb9c59"
server_key_version: "encryptionkey@0"
I20260810 18:41:39.178989 9036 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260810 18:41:39.183740 9063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:39.185786 9036 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260810 18:41:39.186050 9036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-1/wal
uuid: "528ecdf5ef8c4a448ec7fb8dfb2f5e40"
format_stamp: "Formatted at 2026-08-10 18:41:39 on dist-test-slave-7kzw"
server_key: "ae510ba51bc474f5ff52d76717e29ba6"
server_key_iv: "b32f02684f4ad3f15d1c22407deb9c59"
server_key_version: "encryptionkey@0"
I20260810 18:41:39.186302 9036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:41:39.205974 9036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:41:39.211995 9036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:41:39.212385 9036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:41:39.213789 9036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:41:39.215497 9036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:41:39.215582 9036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:39.215684 9036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:41:39.215736 9036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:39.257097 9036 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.194:32911
I20260810 18:41:39.257154 9232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.194:32911 every 8 connection(s)
I20260810 18:41:39.258875 9036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
I20260810 18:41:39.263023 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 9036
I20260810 18:41:39.263247 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance
I20260810 18:41:39.263911 5011 external_mini_cluster.cc:1428] Setting key 847b218f31ee5edfd578fd4d3dc8b18c
Aug 10 18:41:39 dist-test-slave-7kzw krb5kdc[8661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387299, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:39.285338 9237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:39.262169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:38995 (local address 127.4.228.254:34529)
0810 18:41:39.262791 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:39.262798 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:39.264030 (+ 1232us) server_negotiation.cc:407] Connection header received
0810 18:41:39.266258 (+ 2228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:39.266262 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:39.266351 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:39.266441 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:39.269108 (+ 2667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:39.269908 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:39.272965 (+ 3057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:39.273511 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:39.273787 (+ 276us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:39.278295 (+ 4508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:39.278302 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:39.278306 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:39.278348 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:39.281683 (+ 3335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:39.282670 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:39.282675 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:39.282678 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:39.282823 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:39.283362 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:39.283366 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:39.283368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:39.283661 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:39.283828 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:39.284641 (+ 813us) server_negotiation.cc:299] Negotiation successful
0810 18:41:39.284899 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":284,"threads_started":1}
I20260810 18:41:39.286536 9238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:39.262828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:34529 (local address 127.4.228.194:38995)
0810 18:41:39.263825 (+ 997us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:39.263883 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:39.265916 (+ 2033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:39.266806 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:39.266820 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:39.267810 (+ 990us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:39.268854 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:39.268889 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:39.270121 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:39.270130 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:39.272726 (+ 2596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:39.272751 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:39.273912 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:39.273923 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:39.274320 (+ 397us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:39.275322 (+ 1002us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:41:39.275363 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:39.277973 (+ 2610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:39.281913 (+ 3940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:39.281923 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:39.281943 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:39.282451 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:39.282966 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:39.282971 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:39.282975 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:39.283218 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:39.283803 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:39.283813 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:39.284227 (+ 414us) client_negotiation.cc:769] Sending connection context
0810 18:41:39.284513 (+ 286us) client_negotiation.cc:240] Negotiation successful
0810 18:41:39.285135 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"thread_start_us":386,"threads_started":1}
I20260810 18:41:39.288305 9233 heartbeater.cc:347] Connected to a master server at 127.4.228.254:34529
I20260810 18:41:39.288738 9233 heartbeater.cc:464] Registering TS with master...
I20260810 18:41:39.289870 9233 heartbeater.cc:511] Master 127.4.228.254:34529 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.4.228.195@KRBTEST.COM; defaulting to no policy
I20260810 18:41:39.292061 8728 ts_manager.cc:194] Registered new tserver with Master: 528ecdf5ef8c4a448ec7fb8dfb2f5e40 (127.4.228.194:32911)
I20260810 18:41:39.293074 8728 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.194@KRBTEST.COM'} at 127.4.228.194:38995
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab.
Entry for principal kudu/127.4.228.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab.
I20260810 18:41:39.326691 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.195:0
--local_ip_for_outbound_sockets=127.4.228.195
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:34529
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:41567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:41:39.706384 9242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:41:39.706895 9242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:41:39.707003 9242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260810 18:41:39.710789 9030 heartbeater.cc:503] Master 127.4.228.254:34529 was elected leader, sending a full tablet report...
W20260810 18:41:39.715830 9242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:41:39.715962 9242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:41:39.716079 9242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.195
I20260810 18:41:39.726552 9242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:41567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:34529
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.195
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:41:39.728049 9242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:41:39.729809 9242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:41:39.745931 9250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:39.745890 9258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:39.745992 9251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:39.746759 9242 server_base.cc:1034] running on GCE node
I20260810 18:41:39.747478 9242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:41:39.748528 9242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:41:39.749780 9242 hybrid_clock.cc:648] HybridClock initialized: now 1786387299749722 us; error 39 us; skew 500 ppm
I20260810 18:41:39.750133 9242 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:41:39 dist-test-slave-7kzw krb5kdc[8661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387299, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:41:39.753927 9242 init.cc:377] Logged in from keytab as kudu/127.4.228.195@KRBTEST.COM (short username kudu)
I20260810 18:41:39.755697 9242 webserver.cc:460] Webserver started at http://127.4.228.195:44665/ using document root <none> and password file <none>
I20260810 18:41:39.756520 9242 fs_manager.cc:359] Metadata directory not provided
I20260810 18:41:39.756642 9242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:41:39.756935 9242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:41:39.760754 9242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-2/data/instance:
uuid: "d567da4bf7304cccb068c8b13bc30795"
format_stamp: "Formatted at 2026-08-10 18:41:39 on dist-test-slave-7kzw"
server_key: "55056ac90c0805e004c372b783b0e108"
server_key_iv: "48dbe96e55998cc58b06c30fea18a4f0"
server_key_version: "encryptionkey@0"
I20260810 18:41:39.761718 9242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance:
uuid: "d567da4bf7304cccb068c8b13bc30795"
format_stamp: "Formatted at 2026-08-10 18:41:39 on dist-test-slave-7kzw"
server_key: "55056ac90c0805e004c372b783b0e108"
server_key_iv: "48dbe96e55998cc58b06c30fea18a4f0"
server_key_version: "encryptionkey@0"
I20260810 18:41:39.768601 9242 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260810 18:41:39.772873 9267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:39.774837 9242 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 18:41:39.775058 9242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-2/wal
uuid: "d567da4bf7304cccb068c8b13bc30795"
format_stamp: "Formatted at 2026-08-10 18:41:39 on dist-test-slave-7kzw"
server_key: "55056ac90c0805e004c372b783b0e108"
server_key_iv: "48dbe96e55998cc58b06c30fea18a4f0"
server_key_version: "encryptionkey@0"
I20260810 18:41:39.775292 9242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:41:39.795962 9242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:41:39.801484 9242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:41:39.801788 9242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:41:39.803254 9242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:41:39.804950 9242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:41:39.805034 9242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:39.805130 9242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:41:39.805176 9242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:39.846694 9242 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.195:39379
I20260810 18:41:39.846714 9429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.195:39379 every 8 connection(s)
I20260810 18:41:39.848249 9242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
I20260810 18:41:39.855399 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 9242
I20260810 18:41:39.855679 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance
I20260810 18:41:39.856290 5011 external_mini_cluster.cc:1428] Setting key 7f2f40e326222fca2ee9589da99acb22
Aug 10 18:41:39 dist-test-slave-7kzw krb5kdc[8661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387299, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:39.877189 9436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:39.851370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:55785 (local address 127.4.228.254:34529)
0810 18:41:39.851991 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:39.851996 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:39.853105 (+ 1109us) server_negotiation.cc:407] Connection header received
0810 18:41:39.855382 (+ 2277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:39.855389 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:39.855507 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:39.855648 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:39.858038 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:39.858988 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:39.860706 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:39.861535 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:39.861701 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:39.867423 (+ 5722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:39.867431 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:39.867442 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:39.867484 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:39.873005 (+ 5521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:39.874222 (+ 1217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:39.874228 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:39.874232 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:39.874430 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:39.875090 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:39.875094 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:39.875096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:39.875400 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:39.875528 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:39.876608 (+ 1080us) server_negotiation.cc:299] Negotiation successful
0810 18:41:39.876821 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":338,"threads_started":1}
I20260810 18:41:39.878222 9438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:39.852050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:34529 (local address 127.4.228.195:55785)
0810 18:41:39.852901 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:39.852954 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:39.855006 (+ 2052us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:39.856179 (+ 1173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:39.856192 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:39.857021 (+ 829us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:39.857842 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:39.857866 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:39.859170 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:39.859174 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:39.860508 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:39.860526 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:39.861824 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:39.861832 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:39.862163 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:39.863157 (+ 994us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:41:39.863206 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:39.867056 (+ 3850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:39.873206 (+ 6150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:39.873217 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:39.873238 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:39.874051 (+ 813us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:39.874578 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:39.874586 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:39.874593 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:39.874975 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:39.875657 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:39.875665 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:39.876117 (+ 452us) client_negotiation.cc:769] Sending connection context
0810 18:41:39.876413 (+ 296us) client_negotiation.cc:240] Negotiation successful
0810 18:41:39.876899 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":336,"threads_started":1}
I20260810 18:41:39.880170 9431 heartbeater.cc:347] Connected to a master server at 127.4.228.254:34529
I20260810 18:41:39.880559 9431 heartbeater.cc:464] Registering TS with master...
I20260810 18:41:39.881547 9431 heartbeater.cc:511] Master 127.4.228.254:34529 requested a full tablet report, sending...
I20260810 18:41:39.883349 8728 ts_manager.cc:194] Registered new tserver with Master: d567da4bf7304cccb068c8b13bc30795 (127.4.228.195:39379)
I20260810 18:41:39.884358 8728 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.195@KRBTEST.COM'} at 127.4.228.195:55785
I20260810 18:41:39.886888 5011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 18:41:39.932354 9436 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:41:39.928755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41068 (local address 127.4.228.254:34529)
0810 18:41:39.929020 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:39.929030 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:39.929578 (+ 548us) server_negotiation.cc:407] Connection header received
0810 18:41:39.929888 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:39.929893 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:39.929978 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:39.930093 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:39.932136 (+ 2043us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41068: BlockingRecv error: recv got EOF from 127.0.0.1:41068 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
I20260810 18:41:39.936201 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 8819
W20260810 18:41:39.940277 5011 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 8819: Not authorized: Client connection negotiation failed: client connection to 127.4.228.193:35569: server requires authentication, but client does not have Kerberos credentials available
I20260810 18:41:39.940557 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 8819
I20260810 18:41:39.969285 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 9036
W20260810 18:41:39.972483 5011 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 9036: Not authorized: Client connection negotiation failed: client connection to 127.4.228.194:32911: server requires authentication, but client does not have Kerberos credentials available
I20260810 18:41:39.972671 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 9036
I20260810 18:41:40.004703 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 9242
W20260810 18:41:40.009248 5011 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 9242: Not authorized: Client connection negotiation failed: client connection to 127.4.228.195:39379: server requires authentication, but client does not have Kerberos credentials available
W20260810 18:41:40.009411 9458 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:41:40.006313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44758 (local address 127.4.228.195:39379)
0810 18:41:40.007307 (+ 994us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:40.007315 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:40.007369 (+ 54us) server_negotiation.cc:407] Connection header received
0810 18:41:40.007443 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:40.007450 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:40.007882 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:40.008119 (+ 237us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:40.009098 (+ 979us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44758: BlockingRecv error: recv got EOF from 127.0.0.1:44758 (error 108)
Metrics: {"server-negotiator.queue_time_us":687,"thread_start_us":499,"threads_started":1}
I20260810 18:41:40.009490 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 9242
I20260810 18:41:40.036731 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 8680
I20260810 18:41:40.038251 8774 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:41:40.147163 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 8680
2026-08-10T18:41:40Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (4314 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:41:40 dist-test-slave-7kzw krb5kdc[9463](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:41:40 dist-test-slave-7kzw krb5kdc[9463](info): set up 2 sockets
Aug 10 18:41:40 dist-test-slave-7kzw krb5kdc[9463](info): commencing operation
krb5kdc: starting...
W20260810 18:41:41.557163 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.354s user 0.006s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:41:41 dist-test-slave-7kzw krb5kdc[9463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:41:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:41:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.228.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
I20260810 18:41:41.827855 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:46621
--webserver_interface=127.4.228.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:43191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:46621
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:41:42.193984 9484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:41:42.194342 9484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:41:42.194540 9484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:41:42.202850 9484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:41:42.202991 9484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:41:42.203040 9484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:41:42.203126 9484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:41:42.203246 9484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:41:42.214123 9484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:43191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:46621
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:46621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:41:42.215770 9484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:41:42.217258 9484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:41:42.230891 9496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:42.230924 9497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:42.230887 9499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:42.232084 9484 server_base.cc:1034] running on GCE node
I20260810 18:41:42.233054 9484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:41:42.234753 9484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:41:42.236034 9484 hybrid_clock.cc:648] HybridClock initialized: now 1786387302235988 us; error 35 us; skew 500 ppm
I20260810 18:41:42.236490 9484 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:41:42 dist-test-slave-7kzw krb5kdc[9463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387302, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:41:42.240352 9484 init.cc:377] Logged in from keytab as kudu/127.4.228.254@KRBTEST.COM (short username kudu)
I20260810 18:41:42.242213 9484 webserver.cc:460] Webserver started at http://127.4.228.254:34033/ using document root <none> and password file <none>
I20260810 18:41:42.243088 9484 fs_manager.cc:359] Metadata directory not provided
I20260810 18:41:42.243257 9484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:41:42.243607 9484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:41:42.247066 9484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "d283dc5a770a48f7bf6bdca34ca87591"
format_stamp: "Formatted at 2026-08-10 18:41:42 on dist-test-slave-7kzw"
server_key: "181b02835502662292ded9273512fc2d"
server_key_iv: "0e42af6ad01d4f22721e15022ec4b065"
server_key_version: "encryptionkey@0"
I20260810 18:41:42.247962 9484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "d283dc5a770a48f7bf6bdca34ca87591"
format_stamp: "Formatted at 2026-08-10 18:41:42 on dist-test-slave-7kzw"
server_key: "181b02835502662292ded9273512fc2d"
server_key_iv: "0e42af6ad01d4f22721e15022ec4b065"
server_key_version: "encryptionkey@0"
I20260810 18:41:42.253695 9484 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 18:41:42.257812 9509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:42.259464 9484 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:41:42.259804 9484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "d283dc5a770a48f7bf6bdca34ca87591"
format_stamp: "Formatted at 2026-08-10 18:41:42 on dist-test-slave-7kzw"
server_key: "181b02835502662292ded9273512fc2d"
server_key_iv: "0e42af6ad01d4f22721e15022ec4b065"
server_key_version: "encryptionkey@0"
I20260810 18:41:42.260043 9484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:41:42.275347 9484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:41:42.280463 9484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:41:42.280817 9484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:41:42.302764 9484 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:46621
I20260810 18:41:42.302857 9591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:46621 every 8 connection(s)
I20260810 18:41:42.304402 9484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:41:42.305923 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 9484
I20260810 18:41:42.306185 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:41:42.307112 5011 external_mini_cluster.cc:1428] Setting key 323128a97f284c08b8f4f30d1f38d607
I20260810 18:41:42.310030 9592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:42.321516 9592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d283dc5a770a48f7bf6bdca34ca87591: Bootstrap starting.
Aug 10 18:41:42 dist-test-slave-7kzw krb5kdc[9463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:42.325407 9592 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d283dc5a770a48f7bf6bdca34ca87591: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:42.326675 9592 log.cc:824] T 00000000000000000000000000000000 P d283dc5a770a48f7bf6bdca34ca87591: Log is configured to *not* fsync() on all Append() calls
I20260810 18:41:42.330663 9592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d283dc5a770a48f7bf6bdca34ca87591: No bootstrap required, opened a new log
I20260810 18:41:42.332659 9597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:42.309423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50518 (local address 127.4.228.254:46621)
0810 18:41:42.310420 (+ 997us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:42.310434 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:42.310543 (+ 109us) server_negotiation.cc:407] Connection header received
0810 18:41:42.312299 (+ 1756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:42.312324 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:42.312768 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:42.313221 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:42.314941 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:42.316311 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:42.317616 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:42.318340 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:42.318575 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:42.323019 (+ 4444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:42.323028 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:42.323043 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:42.323089 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:42.326925 (+ 3836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:42.328136 (+ 1211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:42.328143 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:42.328151 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:42.328457 (+ 306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:42.328959 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:42.328964 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:42.328967 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:42.329795 (+ 828us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:42.330127 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:42.330667 (+ 540us) server_negotiation.cc:299] Negotiation successful
0810 18:41:42.331184 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":714,"thread_start_us":428,"threads_started":1}
I20260810 18:41:42.337734 5011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:41:42.340085 9592 raft_consensus.cc:348] T 00000000000000000000000000000000 P d283dc5a770a48f7bf6bdca34ca87591 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d283dc5a770a48f7bf6bdca34ca87591" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46621 } }
I20260810 18:41:42.340451 9592 raft_consensus.cc:374] T 00000000000000000000000000000000 P d283dc5a770a48f7bf6bdca34ca87591 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:42.340608 9592 raft_consensus.cc:729] T 00000000000000000000000000000000 P d283dc5a770a48f7bf6bdca34ca87591 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d283dc5a770a48f7bf6bdca34ca87591, State: Initialized, Role: FOLLOWER
I20260810 18:41:42.341287 9592 consensus_queue.cc:260] T 00000000000000000000000000000000 P d283dc5a770a48f7bf6bdca34ca87591 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d283dc5a770a48f7bf6bdca34ca87591" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46621 } }
I20260810 18:41:42.341538 9592 raft_consensus.cc:388] T 00000000000000000000000000000000 P d283dc5a770a48f7bf6bdca34ca87591 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:41:42.341684 9592 raft_consensus.cc:482] T 00000000000000000000000000000000 P d283dc5a770a48f7bf6bdca34ca87591 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:41:42.341879 9592 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d283dc5a770a48f7bf6bdca34ca87591 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:42.344233 9592 raft_consensus.cc:504] T 00000000000000000000000000000000 P d283dc5a770a48f7bf6bdca34ca87591 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d283dc5a770a48f7bf6bdca34ca87591" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46621 } }
I20260810 18:41:42.344794 9592 leader_election.cc:302] T 00000000000000000000000000000000 P d283dc5a770a48f7bf6bdca34ca87591 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d283dc5a770a48f7bf6bdca34ca87591; no voters:
I20260810 18:41:42.345461 9592 leader_election.cc:288] T 00000000000000000000000000000000 P d283dc5a770a48f7bf6bdca34ca87591 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:41:42.345703 9599 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d283dc5a770a48f7bf6bdca34ca87591 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:41:42.346903 9599 raft_consensus.cc:686] T 00000000000000000000000000000000 P d283dc5a770a48f7bf6bdca34ca87591 [term 1 LEADER]: Becoming Leader. State: Replica: d283dc5a770a48f7bf6bdca34ca87591, State: Running, Role: LEADER
I20260810 18:41:42.347636 9599 consensus_queue.cc:237] T 00000000000000000000000000000000 P d283dc5a770a48f7bf6bdca34ca87591 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d283dc5a770a48f7bf6bdca34ca87591" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46621 } }
I20260810 18:41:42.347924 9592 sys_catalog.cc:564] T 00000000000000000000000000000000 P d283dc5a770a48f7bf6bdca34ca87591 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:41:42.351122 9601 sys_catalog.cc:455] T 00000000000000000000000000000000 P d283dc5a770a48f7bf6bdca34ca87591 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d283dc5a770a48f7bf6bdca34ca87591. Latest consensus state: current_term: 1 leader_uuid: "d283dc5a770a48f7bf6bdca34ca87591" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d283dc5a770a48f7bf6bdca34ca87591" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46621 } } }
I20260810 18:41:42.351163 9600 sys_catalog.cc:455] T 00000000000000000000000000000000 P d283dc5a770a48f7bf6bdca34ca87591 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d283dc5a770a48f7bf6bdca34ca87591" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d283dc5a770a48f7bf6bdca34ca87591" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46621 } } }
I20260810 18:41:42.351604 9601 sys_catalog.cc:458] T 00000000000000000000000000000000 P d283dc5a770a48f7bf6bdca34ca87591 [sys.catalog]: This master's current role is: LEADER
I20260810 18:41:42.351629 9600 sys_catalog.cc:458] T 00000000000000000000000000000000 P d283dc5a770a48f7bf6bdca34ca87591 [sys.catalog]: This master's current role is: LEADER
I20260810 18:41:42.352718 9613 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:41:42.369462 9613 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:41:42.400177 9613 catalog_manager.cc:1347] Generated new cluster ID: 2c2fb7122e334e0c8591d688a3bf21d5
I20260810 18:41:42.400352 9613 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:41:42.419651 9613 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:41:42.421862 9613 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:41:42.437955 9613 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d283dc5a770a48f7bf6bdca34ca87591: Generated new TSK 0
I20260810 18:41:42.439189 9613 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:41:42.441219 9597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:42.424556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50520 (local address 127.4.228.254:46621)
0810 18:41:42.424851 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:42.424862 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:42.432266 (+ 7404us) server_negotiation.cc:407] Connection header received
0810 18:41:42.432532 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:42.432540 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:42.432649 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:42.432785 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:42.434052 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:42.435020 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:42.436886 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:42.437573 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:42.437734 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:42.439309 (+ 1575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:42.439316 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:42.439321 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:41:42.440253 (+ 932us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:42.440309 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:41:42.440341 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:42.440485 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:42.440671 (+ 186us) server_negotiation.cc:299] Negotiation successful
0810 18:41:42.440791 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260810 18:41:42.450955 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 9484
I20260810 18:41:42.452713 9587 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:41:42.548993 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 9484
2026-08-10T18:41:42Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2397 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:41:42 dist-test-slave-7kzw krb5kdc[9646](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:41:42 dist-test-slave-7kzw krb5kdc[9646](info): set up 2 sockets
Aug 10 18:41:42 dist-test-slave-7kzw krb5kdc[9646](info): commencing operation
krb5kdc: starting...
W20260810 18:41:43.929368 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.329s user 0.000s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:41:44 dist-test-slave-7kzw krb5kdc[9646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:41:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:41:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.228.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
I20260810 18:41:44.195436 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:40025
--webserver_interface=127.4.228.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:46763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:40025
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:41:44.568907 9664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:41:44.569259 9664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:41:44.569438 9664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:41:44.577556 9664 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:41:44.577709 9664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:41:44.577860 9664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:41:44.577961 9664 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:41:44.578054 9664 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:41:44.588687 9664 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:46763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:40025
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:40025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:41:44.590246 9664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:41:44.591840 9664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:41:44.605324 9672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:44.605284 9674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:44.605355 9678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:44.606357 9664 server_base.cc:1034] running on GCE node
I20260810 18:41:44.607343 9664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:41:44.609107 9664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:41:44.610395 9664 hybrid_clock.cc:648] HybridClock initialized: now 1786387304610318 us; error 69 us; skew 500 ppm
I20260810 18:41:44.610919 9664 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:41:44 dist-test-slave-7kzw krb5kdc[9646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387304, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:41:44.614861 9664 init.cc:377] Logged in from keytab as kudu/127.4.228.254@KRBTEST.COM (short username kudu)
I20260810 18:41:44.616694 9664 webserver.cc:460] Webserver started at http://127.4.228.254:33881/ using document root <none> and password file <none>
I20260810 18:41:44.617518 9664 fs_manager.cc:359] Metadata directory not provided
I20260810 18:41:44.617638 9664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:41:44.617986 9664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:41:44.621884 9664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "36281d0fcf114d2abeeaf092df7a6072"
format_stamp: "Formatted at 2026-08-10 18:41:44 on dist-test-slave-7kzw"
server_key: "6404bf4d302936e2a9e193e3dff083ee"
server_key_iv: "c9a6c2b82fbd61410f703844ecbfb0c1"
server_key_version: "encryptionkey@0"
I20260810 18:41:44.622884 9664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "36281d0fcf114d2abeeaf092df7a6072"
format_stamp: "Formatted at 2026-08-10 18:41:44 on dist-test-slave-7kzw"
server_key: "6404bf4d302936e2a9e193e3dff083ee"
server_key_iv: "c9a6c2b82fbd61410f703844ecbfb0c1"
server_key_version: "encryptionkey@0"
I20260810 18:41:44.628978 9664 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260810 18:41:44.633549 9689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:44.635339 9664 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 18:41:44.635560 9664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "36281d0fcf114d2abeeaf092df7a6072"
format_stamp: "Formatted at 2026-08-10 18:41:44 on dist-test-slave-7kzw"
server_key: "6404bf4d302936e2a9e193e3dff083ee"
server_key_iv: "c9a6c2b82fbd61410f703844ecbfb0c1"
server_key_version: "encryptionkey@0"
I20260810 18:41:44.635805 9664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:41:44.657126 9664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:41:44.662323 9664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:41:44.662724 9664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:41:44.684516 9664 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:40025
I20260810 18:41:44.684548 9767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:40025 every 8 connection(s)
I20260810 18:41:44.686095 9664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:41:44.690632 9768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:44.693547 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 9664
I20260810 18:41:44.693846 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:41:44.694808 5011 external_mini_cluster.cc:1428] Setting key 4e2e95671a031cc883cbb9c9f5daa9c4
I20260810 18:41:44.705753 9768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36281d0fcf114d2abeeaf092df7a6072: Bootstrap starting.
I20260810 18:41:44.709457 9768 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36281d0fcf114d2abeeaf092df7a6072: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:44.710743 9768 log.cc:824] T 00000000000000000000000000000000 P 36281d0fcf114d2abeeaf092df7a6072: Log is configured to *not* fsync() on all Append() calls
Aug 10 18:41:44 dist-test-slave-7kzw krb5kdc[9646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:44.714919 9768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36281d0fcf114d2abeeaf092df7a6072: No bootstrap required, opened a new log
I20260810 18:41:44.722927 9772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:44.697523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58592 (local address 127.4.228.254:40025)
0810 18:41:44.698651 (+ 1128us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:44.698667 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:44.698721 (+ 54us) server_negotiation.cc:407] Connection header received
0810 18:41:44.700541 (+ 1820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:44.700569 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:44.701053 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:44.701479 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:44.703494 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:44.705071 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:44.706790 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:44.707627 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:44.707898 (+ 271us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:44.713136 (+ 5238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:44.713145 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:44.713177 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:44.713222 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:44.717142 (+ 3920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:44.718258 (+ 1116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:44.718267 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:44.718280 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:44.718717 (+ 437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:44.719277 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:44.719282 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:44.719285 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:44.720054 (+ 769us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:44.720382 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:44.720926 (+ 544us) server_negotiation.cc:299] Negotiation successful
0810 18:41:44.721520 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":792,"thread_start_us":498,"threads_started":1}
I20260810 18:41:44.725211 9768 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36281d0fcf114d2abeeaf092df7a6072 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36281d0fcf114d2abeeaf092df7a6072" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 40025 } }
I20260810 18:41:44.725660 9768 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36281d0fcf114d2abeeaf092df7a6072 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:44.725804 9768 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36281d0fcf114d2abeeaf092df7a6072 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36281d0fcf114d2abeeaf092df7a6072, State: Initialized, Role: FOLLOWER
I20260810 18:41:44.726606 9768 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36281d0fcf114d2abeeaf092df7a6072 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36281d0fcf114d2abeeaf092df7a6072" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 40025 } }
I20260810 18:41:44.726883 9768 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36281d0fcf114d2abeeaf092df7a6072 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:41:44.727057 9768 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36281d0fcf114d2abeeaf092df7a6072 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:41:44.727299 9768 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36281d0fcf114d2abeeaf092df7a6072 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:44.729846 9768 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36281d0fcf114d2abeeaf092df7a6072 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36281d0fcf114d2abeeaf092df7a6072" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 40025 } }
I20260810 18:41:44.730517 9768 leader_election.cc:302] T 00000000000000000000000000000000 P 36281d0fcf114d2abeeaf092df7a6072 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36281d0fcf114d2abeeaf092df7a6072; no voters:
I20260810 18:41:44.731272 9768 leader_election.cc:288] T 00000000000000000000000000000000 P 36281d0fcf114d2abeeaf092df7a6072 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:41:44.731459 9775 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36281d0fcf114d2abeeaf092df7a6072 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:41:44.732797 9775 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36281d0fcf114d2abeeaf092df7a6072 [term 1 LEADER]: Becoming Leader. State: Replica: 36281d0fcf114d2abeeaf092df7a6072, State: Running, Role: LEADER
I20260810 18:41:44.733407 9775 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36281d0fcf114d2abeeaf092df7a6072 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36281d0fcf114d2abeeaf092df7a6072" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 40025 } }
I20260810 18:41:44.733891 9768 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36281d0fcf114d2abeeaf092df7a6072 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:41:44.736624 9777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36281d0fcf114d2abeeaf092df7a6072 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36281d0fcf114d2abeeaf092df7a6072. Latest consensus state: current_term: 1 leader_uuid: "36281d0fcf114d2abeeaf092df7a6072" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36281d0fcf114d2abeeaf092df7a6072" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 40025 } } }
I20260810 18:41:44.737129 9777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36281d0fcf114d2abeeaf092df7a6072 [sys.catalog]: This master's current role is: LEADER
I20260810 18:41:44.736624 9776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36281d0fcf114d2abeeaf092df7a6072 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36281d0fcf114d2abeeaf092df7a6072" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36281d0fcf114d2abeeaf092df7a6072" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 40025 } } }
I20260810 18:41:44.737766 9776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36281d0fcf114d2abeeaf092df7a6072 [sys.catalog]: This master's current role is: LEADER
I20260810 18:41:44.744199 9781 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:41:44.749147 9781 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:41:44.759323 9781 catalog_manager.cc:1347] Generated new cluster ID: 3c4e0cd1e9b043b7b257c09b80cb7b64
I20260810 18:41:44.759485 9781 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:41:44.779038 9781 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:41:44.781525 9781 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:41:44.801044 9781 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36281d0fcf114d2abeeaf092df7a6072: Generated new TSK 0
I20260810 18:41:44.802008 9781 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.228.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab.
Entry for principal kudu/127.4.228.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab.
I20260810 18:41:44.890848 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.193:0
--local_ip_for_outbound_sockets=127.4.228.193
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:40025
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:46763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:41:45.293818 9805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:41:45.294169 9805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:41:45.294269 9805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:41:45.302042 9805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:41:45.302170 9805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:41:45.302286 9805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.193
I20260810 18:41:45.312659 9805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:46763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:40025
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.193
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:41:45.314090 9805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:41:45.315852 9805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:41:45.330443 9813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:45.330411 9814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:45.330672 9816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:45.331478 9805 server_base.cc:1034] running on GCE node
I20260810 18:41:45.332266 9805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:41:45.333420 9805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:41:45.334678 9805 hybrid_clock.cc:648] HybridClock initialized: now 1786387305334598 us; error 76 us; skew 500 ppm
I20260810 18:41:45.335055 9805 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:41:45 dist-test-slave-7kzw krb5kdc[9646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387305, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:41:45.338657 9805 init.cc:377] Logged in from keytab as kudu/127.4.228.193@KRBTEST.COM (short username kudu)
I20260810 18:41:45.340657 9805 webserver.cc:460] Webserver started at http://127.4.228.193:39189/ using document root <none> and password file <none>
I20260810 18:41:45.341523 9805 fs_manager.cc:359] Metadata directory not provided
I20260810 18:41:45.341681 9805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:41:45.342090 9805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:41:45.345481 9805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-0/data/instance:
uuid: "ce29192e6b194e9caeb14a5255dbf781"
format_stamp: "Formatted at 2026-08-10 18:41:45 on dist-test-slave-7kzw"
server_key: "0782995ea9f376e013f5c68a0bfd89ce"
server_key_iv: "cfdecf65464d5cb069d498782f29956e"
server_key_version: "encryptionkey@0"
I20260810 18:41:45.346405 9805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance:
uuid: "ce29192e6b194e9caeb14a5255dbf781"
format_stamp: "Formatted at 2026-08-10 18:41:45 on dist-test-slave-7kzw"
server_key: "0782995ea9f376e013f5c68a0bfd89ce"
server_key_iv: "cfdecf65464d5cb069d498782f29956e"
server_key_version: "encryptionkey@0"
I20260810 18:41:45.352463 9805 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260810 18:41:45.356729 9831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:45.358300 9805 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:41:45.358579 9805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-0/wal
uuid: "ce29192e6b194e9caeb14a5255dbf781"
format_stamp: "Formatted at 2026-08-10 18:41:45 on dist-test-slave-7kzw"
server_key: "0782995ea9f376e013f5c68a0bfd89ce"
server_key_iv: "cfdecf65464d5cb069d498782f29956e"
server_key_version: "encryptionkey@0"
I20260810 18:41:45.358811 9805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:41:45.381606 9805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:41:45.386967 9805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:41:45.387370 9805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:41:45.388583 9805 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:41:45.390249 9805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:41:45.390385 9805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:45.390578 9805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:41:45.390666 9805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:45.430721 9805 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.193:44211
I20260810 18:41:45.430873 10005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.193:44211 every 8 connection(s)
I20260810 18:41:45.432443 9805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
I20260810 18:41:45.434340 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 9805
I20260810 18:41:45.434628 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance
I20260810 18:41:45.435431 5011 external_mini_cluster.cc:1428] Setting key 2da8b37483d95cca39dfeca021d7a3e4
Aug 10 18:41:45 dist-test-slave-7kzw krb5kdc[9646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387305, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:45.455647 10009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:45.435835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:47271 (local address 127.4.228.254:40025)
0810 18:41:45.436422 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:45.436427 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:45.437644 (+ 1217us) server_negotiation.cc:407] Connection header received
0810 18:41:45.439810 (+ 2166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:45.439816 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:45.439908 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:45.440039 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:45.442173 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:45.442996 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:45.444653 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:45.445075 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:45.445230 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:45.449096 (+ 3866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:45.449102 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:45.449106 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:45.449143 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:45.452003 (+ 2860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:45.452873 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:45.452882 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:45.452890 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:45.453023 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:45.453523 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:45.453529 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:45.453532 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:45.453859 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:45.454030 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:45.454993 (+ 963us) server_negotiation.cc:299] Negotiation successful
0810 18:41:45.455247 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":290,"threads_started":1}
I20260810 18:41:45.456789 10010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:45.436531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:40025 (local address 127.4.228.193:47271)
0810 18:41:45.437485 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:45.437547 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:45.439472 (+ 1925us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:45.440391 (+ 919us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:45.440403 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:45.441173 (+ 770us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:45.441971 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:45.441994 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:45.443173 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:45.443186 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:45.444480 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:45.444493 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:45.445351 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:45.445361 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:45.445612 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:45.446376 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:41:45.446402 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:45.448817 (+ 2415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:45.452188 (+ 3371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:45.452199 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:45.452221 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:45.452730 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:45.453151 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:45.453156 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:45.453160 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:45.453397 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:45.454005 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:45.454015 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:45.454465 (+ 450us) client_negotiation.cc:769] Sending connection context
0810 18:41:45.454881 (+ 416us) client_negotiation.cc:240] Negotiation successful
0810 18:41:45.455514 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":381,"threads_started":1}
WARNING: no policy specified for kudu/127.4.228.194@KRBTEST.COM; defaulting to no policy
I20260810 18:41:45.458889 10006 heartbeater.cc:347] Connected to a master server at 127.4.228.254:40025
I20260810 18:41:45.459299 10006 heartbeater.cc:464] Registering TS with master...
I20260810 18:41:45.460296 10006 heartbeater.cc:511] Master 127.4.228.254:40025 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.194@KRBTEST.COM" created.
I20260810 18:41:45.463495 9719 ts_manager.cc:194] Registered new tserver with Master: ce29192e6b194e9caeb14a5255dbf781 (127.4.228.193:44211)
I20260810 18:41:45.465116 9719 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.193@KRBTEST.COM'} at 127.4.228.193:47271
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab.
Entry for principal kudu/127.4.228.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab.
I20260810 18:41:45.499246 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.194:0
--local_ip_for_outbound_sockets=127.4.228.194
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:40025
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:46763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:41:45.870242 10015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:41:45.870642 10015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:41:45.870838 10015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:41:45.878163 10015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:41:45.878336 10015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:41:45.878552 10015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.194
I20260810 18:41:45.888598 10015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:46763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:40025
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.194
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:41:45.890161 10015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:41:45.891626 10015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:41:45.905550 10024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:45.905652 10023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:45.905608 10026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:45.906912 10015 server_base.cc:1034] running on GCE node
I20260810 18:41:45.907642 10015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:41:45.908782 10015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:41:45.910053 10015 hybrid_clock.cc:648] HybridClock initialized: now 1786387305909995 us; error 53 us; skew 500 ppm
I20260810 18:41:45.910429 10015 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:41:45 dist-test-slave-7kzw krb5kdc[9646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387305, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:41:45.918920 10015 init.cc:377] Logged in from keytab as kudu/127.4.228.194@KRBTEST.COM (short username kudu)
I20260810 18:41:45.920799 10015 webserver.cc:460] Webserver started at http://127.4.228.194:38241/ using document root <none> and password file <none>
I20260810 18:41:45.921659 10015 fs_manager.cc:359] Metadata directory not provided
I20260810 18:41:45.921788 10015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:41:45.922111 10015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:41:45.925696 10015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-1/data/instance:
uuid: "d44cb13f60004cf7ad4b7031158e9c3a"
format_stamp: "Formatted at 2026-08-10 18:41:45 on dist-test-slave-7kzw"
server_key: "f55aa6bcc8960dbdf04eab1d27a08481"
server_key_iv: "11d6bb757f12cb95820e673f38eb5d61"
server_key_version: "encryptionkey@0"
I20260810 18:41:45.926697 10015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance:
uuid: "d44cb13f60004cf7ad4b7031158e9c3a"
format_stamp: "Formatted at 2026-08-10 18:41:45 on dist-test-slave-7kzw"
server_key: "f55aa6bcc8960dbdf04eab1d27a08481"
server_key_iv: "11d6bb757f12cb95820e673f38eb5d61"
server_key_version: "encryptionkey@0"
I20260810 18:41:45.932819 10015 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 18:41:45.937160 10039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:45.938866 10015 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260810 18:41:45.939033 10015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-1/wal
uuid: "d44cb13f60004cf7ad4b7031158e9c3a"
format_stamp: "Formatted at 2026-08-10 18:41:45 on dist-test-slave-7kzw"
server_key: "f55aa6bcc8960dbdf04eab1d27a08481"
server_key_iv: "11d6bb757f12cb95820e673f38eb5d61"
server_key_version: "encryptionkey@0"
I20260810 18:41:45.939249 10015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:41:45.953557 10015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:41:45.958388 10015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:41:45.958724 10015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:41:45.959714 10015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:41:45.961144 10015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:41:45.961222 10015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:45.961313 10015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:41:45.961364 10015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:45.999426 10015 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.194:36907
I20260810 18:41:45.999532 10204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.194:36907 every 8 connection(s)
I20260810 18:41:46.001034 10015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
I20260810 18:41:46.006162 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 10015
I20260810 18:41:46.006417 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance
I20260810 18:41:46.007004 5011 external_mini_cluster.cc:1428] Setting key df708c96e2bc2797da6481370d8aaeab
Aug 10 18:41:46 dist-test-slave-7kzw krb5kdc[9646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387305, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:46.029574 10208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:46.004138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:54151 (local address 127.4.228.254:40025)
0810 18:41:46.005105 (+ 967us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:46.005116 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:46.006611 (+ 1495us) server_negotiation.cc:407] Connection header received
0810 18:41:46.008864 (+ 2253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:46.008870 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:46.008973 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:46.009101 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:46.011543 (+ 2442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:46.012412 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:46.015169 (+ 2757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:46.015676 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:46.015808 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:46.020876 (+ 5068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:46.020887 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:46.020896 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:46.020951 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:46.025178 (+ 4227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:46.026267 (+ 1089us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:46.026275 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:46.026282 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:46.026455 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:46.027128 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:46.027137 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:46.027143 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:46.027632 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:46.027803 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:46.028720 (+ 917us) server_negotiation.cc:299] Negotiation successful
0810 18:41:46.029034 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":371,"threads_started":1}
I20260810 18:41:46.030301 10209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:46.005110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:40025 (local address 127.4.228.194:54151)
0810 18:41:46.006361 (+ 1251us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:46.006436 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:46.008549 (+ 2113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:46.009408 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:46.009421 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:46.010299 (+ 878us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:46.011290 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:46.011323 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:46.012598 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:46.012606 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:46.014957 (+ 2351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:46.014977 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:46.015970 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:46.015980 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:46.016332 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:46.017402 (+ 1070us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:41:46.017441 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:46.020475 (+ 3034us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:46.025458 (+ 4983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:46.025469 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:46.025487 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:46.026082 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:46.026675 (+ 593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:46.026680 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:46.026683 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:46.026929 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:46.027826 (+ 897us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:46.027833 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:46.028248 (+ 415us) client_negotiation.cc:769] Sending connection context
0810 18:41:46.028510 (+ 262us) client_negotiation.cc:240] Negotiation successful
0810 18:41:46.028994 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":882,"thread_start_us":562,"threads_started":1}
I20260810 18:41:46.032094 10205 heartbeater.cc:347] Connected to a master server at 127.4.228.254:40025
I20260810 18:41:46.032505 10205 heartbeater.cc:464] Registering TS with master...
I20260810 18:41:46.033588 10205 heartbeater.cc:511] Master 127.4.228.254:40025 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.4.228.195@KRBTEST.COM; defaulting to no policy
I20260810 18:41:46.035753 9719 ts_manager.cc:194] Registered new tserver with Master: d44cb13f60004cf7ad4b7031158e9c3a (127.4.228.194:36907)
I20260810 18:41:46.036898 9719 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.194@KRBTEST.COM'} at 127.4.228.194:54151
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab.
Entry for principal kudu/127.4.228.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab.
I20260810 18:41:46.070888 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.195:0
--local_ip_for_outbound_sockets=127.4.228.195
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:40025
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:46763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:41:46.440025 10213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:41:46.440332 10213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:41:46.440431 10213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:41:46.448264 10213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:41:46.448395 10213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:41:46.448513 10213 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.195
I20260810 18:41:46.458732 10213 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:46763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:40025
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.195
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:41:46.460160 10213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:41:46.461594 10213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 18:41:46.468945 10006 heartbeater.cc:503] Master 127.4.228.254:40025 was elected leader, sending a full tablet report...
W20260810 18:41:46.478549 10221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:46.478621 10223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:46.478726 10213 server_base.cc:1034] running on GCE node
W20260810 18:41:46.478577 10225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:46.479784 10213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:41:46.480899 10213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:41:46.482143 10213 hybrid_clock.cc:648] HybridClock initialized: now 1786387306482077 us; error 46 us; skew 500 ppm
I20260810 18:41:46.482620 10213 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:41:46 dist-test-slave-7kzw krb5kdc[9646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387306, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:41:46.486204 10213 init.cc:377] Logged in from keytab as kudu/127.4.228.195@KRBTEST.COM (short username kudu)
I20260810 18:41:46.487915 10213 webserver.cc:460] Webserver started at http://127.4.228.195:43499/ using document root <none> and password file <none>
I20260810 18:41:46.488667 10213 fs_manager.cc:359] Metadata directory not provided
I20260810 18:41:46.488777 10213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:41:46.489050 10213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:41:46.492276 10213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-2/data/instance:
uuid: "6f572ecc20074004862212745283ff2a"
format_stamp: "Formatted at 2026-08-10 18:41:46 on dist-test-slave-7kzw"
server_key: "6a118bcb58de3d7aed89a4317664d0bf"
server_key_iv: "223fc424097e0bc91666a4c1ef3de719"
server_key_version: "encryptionkey@0"
I20260810 18:41:46.493059 10213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance:
uuid: "6f572ecc20074004862212745283ff2a"
format_stamp: "Formatted at 2026-08-10 18:41:46 on dist-test-slave-7kzw"
server_key: "6a118bcb58de3d7aed89a4317664d0bf"
server_key_iv: "223fc424097e0bc91666a4c1ef3de719"
server_key_version: "encryptionkey@0"
I20260810 18:41:46.498811 10213 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 18:41:46.502730 10234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:46.504311 10213 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:41:46.504477 10213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-2/wal
uuid: "6f572ecc20074004862212745283ff2a"
format_stamp: "Formatted at 2026-08-10 18:41:46 on dist-test-slave-7kzw"
server_key: "6a118bcb58de3d7aed89a4317664d0bf"
server_key_iv: "223fc424097e0bc91666a4c1ef3de719"
server_key_version: "encryptionkey@0"
I20260810 18:41:46.504658 10213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:41:46.526103 10213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:41:46.531281 10213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:41:46.531607 10213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:41:46.532727 10213 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:41:46.534407 10213 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:41:46.534556 10213 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260810 18:41:46.534721 10213 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:41:46.534794 10213 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:46.574055 10213 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.195:44787
I20260810 18:41:46.574093 10408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.195:44787 every 8 connection(s)
I20260810 18:41:46.575474 10213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
I20260810 18:41:46.578104 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 10213
I20260810 18:41:46.578372 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance
I20260810 18:41:46.579109 5011 external_mini_cluster.cc:1428] Setting key 403ba1e172f41750c7a38e1b5c4efa95
Aug 10 18:41:46 dist-test-slave-7kzw krb5kdc[9646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387306, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:46.599957 10412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:46.578755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:34205 (local address 127.4.228.254:40025)
0810 18:41:46.579303 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:46.579311 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:46.580607 (+ 1296us) server_negotiation.cc:407] Connection header received
0810 18:41:46.583160 (+ 2553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:46.583165 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:46.583274 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:46.583401 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:46.585062 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:46.585850 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:46.587587 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:46.588014 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:46.588174 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:46.592494 (+ 4320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:46.592500 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:46.592504 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:46.592542 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:46.596119 (+ 3577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:46.597065 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:46.597070 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:46.597072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:46.597228 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:46.597822 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:46.597827 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:46.597829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:46.598159 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:46.598323 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:46.599411 (+ 1088us) server_negotiation.cc:299] Negotiation successful
0810 18:41:46.599633 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":266,"threads_started":1}
I20260810 18:41:46.601106 10415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:46.579361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:40025 (local address 127.4.228.195:34205)
0810 18:41:46.580431 (+ 1070us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:46.580497 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:46.582795 (+ 2298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:46.583675 (+ 880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:46.583683 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:46.584272 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:46.584882 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:46.584901 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:46.586019 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:46.586025 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:46.587428 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:46.587443 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:46.588286 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:46.588294 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:46.588550 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:46.589351 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:41:46.589383 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:46.592222 (+ 2839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:46.596297 (+ 4075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:46.596307 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:46.596323 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:46.596899 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:46.597374 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:46.597379 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:46.597381 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:46.597616 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:46.598359 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:46.598368 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:46.598885 (+ 517us) client_negotiation.cc:769] Sending connection context
0810 18:41:46.599248 (+ 363us) client_negotiation.cc:240] Negotiation successful
0810 18:41:46.599750 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"thread_start_us":465,"threads_started":1}
I20260810 18:41:46.602937 10410 heartbeater.cc:347] Connected to a master server at 127.4.228.254:40025
I20260810 18:41:46.603313 10410 heartbeater.cc:464] Registering TS with master...
I20260810 18:41:46.604174 10410 heartbeater.cc:511] Master 127.4.228.254:40025 requested a full tablet report, sending...
I20260810 18:41:46.605983 9719 ts_manager.cc:194] Registered new tserver with Master: 6f572ecc20074004862212745283ff2a (127.4.228.195:44787)
I20260810 18:41:46.607254 9719 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.195@KRBTEST.COM'} at 127.4.228.195:34205
I20260810 18:41:46.609127 5011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 18:41:46 dist-test-slave-7kzw krb5kdc[9646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387306, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 10 18:41:46 dist-test-slave-7kzw krb5kdc[9646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387306, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:46.675732 10412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:46.658595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58602 (local address 127.4.228.254:40025)
0810 18:41:46.658850 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:46.658859 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:46.659478 (+ 619us) server_negotiation.cc:407] Connection header received
0810 18:41:46.659672 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:46.659676 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:46.659772 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:46.659871 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:46.661671 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:46.662729 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:46.664303 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:46.664785 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:46.664908 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:46.669421 (+ 4513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:46.669427 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:46.669431 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:46.669471 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:46.672695 (+ 3224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:46.673491 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:46.673496 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:46.673498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:46.673640 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:46.674080 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:46.674085 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:46.674088 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:46.674451 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:46.674792 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:46.675124 (+ 332us) server_negotiation.cc:299] Negotiation successful
0810 18:41:46.675378 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260810 18:41:46.676388 9719 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:58602
I20260810 18:41:46.678090 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 9805
Aug 10 18:41:46 dist-test-slave-7kzw krb5kdc[9646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387306, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.4.228.193@KRBTEST.COM
I20260810 18:41:46.698379 10430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:46.679692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39362 (local address 127.4.228.193:44211)
0810 18:41:46.680351 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:46.680359 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:46.680385 (+ 26us) server_negotiation.cc:407] Connection header received
0810 18:41:46.680445 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:46.680452 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:46.680711 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:46.680843 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:46.682088 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:46.683399 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:46.685050 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:46.686063 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:46.686221 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:46.690888 (+ 4667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:46.690906 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:46.690931 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:46.690981 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:46.695199 (+ 4218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:46.696110 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:46.696118 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:46.696121 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:46.696270 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:46.696707 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:46.696711 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:46.696713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:46.697134 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:46.697385 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:46.697660 (+ 275us) server_negotiation.cc:299] Negotiation successful
0810 18:41:46.697995 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":300,"threads_started":1}
W20260810 18:41:46.699846 9997 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:39362
W20260810 18:41:46.701171 5011 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 9805: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:41:46.701362 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 9805
I20260810 18:41:46.730183 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 10015
Aug 10 18:41:46 dist-test-slave-7kzw krb5kdc[9646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387306, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.4.228.194@KRBTEST.COM
I20260810 18:41:46.749804 10431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:46.731361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55210 (local address 127.4.228.194:36907)
0810 18:41:46.732074 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:46.732082 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:46.732113 (+ 31us) server_negotiation.cc:407] Connection header received
0810 18:41:46.732182 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:46.732189 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:46.732422 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:46.732579 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:46.734151 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:46.735411 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:46.736847 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:46.737407 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:46.737513 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:46.743176 (+ 5663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:46.743198 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:46.743215 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:46.743255 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:46.746820 (+ 3565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:46.747590 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:46.747602 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:46.747606 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:46.747764 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:46.748205 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:46.748208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:46.748211 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:46.748632 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:46.748902 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:46.749247 (+ 345us) server_negotiation.cc:299] Negotiation successful
0810 18:41:46.749485 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":387,"threads_started":1}
W20260810 18:41:46.751791 10197 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:55210
W20260810 18:41:46.753196 5011 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 10015: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:41:46.753435 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 10015
I20260810 18:41:46.783696 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 10213
Aug 10 18:41:46 dist-test-slave-7kzw krb5kdc[9646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387306, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.4.228.195@KRBTEST.COM
I20260810 18:41:46.802450 10432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:46.784920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43318 (local address 127.4.228.195:44787)
0810 18:41:46.785535 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:46.785543 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:46.785575 (+ 32us) server_negotiation.cc:407] Connection header received
0810 18:41:46.785639 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:46.785647 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:46.785906 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:46.786045 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:46.788072 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:46.789206 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:46.790595 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:46.791221 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:46.791332 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:46.796232 (+ 4900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:46.796249 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:46.796271 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:46.796313 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:46.799540 (+ 3227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:46.800347 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:46.800357 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:46.800359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:46.800494 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:46.800932 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:46.800935 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:46.800937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:46.801317 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:46.801558 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:46.801920 (+ 362us) server_negotiation.cc:299] Negotiation successful
0810 18:41:46.802139 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":294,"threads_started":1}
W20260810 18:41:46.803987 10399 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:43318
W20260810 18:41:46.805344 5011 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 10213: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:41:46.805536 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 10213
I20260810 18:41:46.833199 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 9664
I20260810 18:41:46.835281 9762 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:41:46.945210 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 9664
2026-08-10T18:41:46Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (4401 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:41:47 dist-test-slave-7kzw krb5kdc[10438](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:41:47 dist-test-slave-7kzw krb5kdc[10438](info): set up 2 sockets
Aug 10 18:41:47 dist-test-slave-7kzw krb5kdc[10438](info): commencing operation
krb5kdc: starting...
W20260810 18:41:48.310729 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.309s user 0.004s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:41:48 dist-test-slave-7kzw krb5kdc[10438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:41:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:41:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.228.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
I20260810 18:41:48.583413 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:32859
--webserver_interface=127.4.228.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:40429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:32859
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:41:48.949267 10457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:41:48.949608 10457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:41:48.949704 10457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:41:48.957314 10457 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:41:48.957453 10457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:41:48.957501 10457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:41:48.957553 10457 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:41:48.957599 10457 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:41:48.968770 10457 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:40429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:32859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:32859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:41:48.970297 10457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:41:48.971814 10457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:41:48.984957 10468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:48.985060 10465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:48.985035 10466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:48.986058 10457 server_base.cc:1034] running on GCE node
I20260810 18:41:48.987018 10457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:41:48.988649 10457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:41:48.989909 10457 hybrid_clock.cc:648] HybridClock initialized: now 1786387308989856 us; error 44 us; skew 500 ppm
I20260810 18:41:48.990351 10457 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:41:48 dist-test-slave-7kzw krb5kdc[10438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387308, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:41:48.994088 10457 init.cc:377] Logged in from keytab as kudu/127.4.228.254@KRBTEST.COM (short username kudu)
I20260810 18:41:48.996167 10457 webserver.cc:460] Webserver started at http://127.4.228.254:38225/ using document root <none> and password file <none>
I20260810 18:41:48.997057 10457 fs_manager.cc:359] Metadata directory not provided
I20260810 18:41:48.997236 10457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:41:48.997639 10457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:41:49.001368 10457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "aaff2299495e42d09ed87a73241e012f"
format_stamp: "Formatted at 2026-08-10 18:41:48 on dist-test-slave-7kzw"
server_key: "f874dd18be96a28c70bf49699a88b88c"
server_key_iv: "30243e535101232c2799581250279cf3"
server_key_version: "encryptionkey@0"
I20260810 18:41:49.002326 10457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "aaff2299495e42d09ed87a73241e012f"
format_stamp: "Formatted at 2026-08-10 18:41:48 on dist-test-slave-7kzw"
server_key: "f874dd18be96a28c70bf49699a88b88c"
server_key_iv: "30243e535101232c2799581250279cf3"
server_key_version: "encryptionkey@0"
I20260810 18:41:49.008903 10457 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 18:41:49.013103 10477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:49.015007 10457 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260810 18:41:49.015259 10457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "aaff2299495e42d09ed87a73241e012f"
format_stamp: "Formatted at 2026-08-10 18:41:48 on dist-test-slave-7kzw"
server_key: "f874dd18be96a28c70bf49699a88b88c"
server_key_iv: "30243e535101232c2799581250279cf3"
server_key_version: "encryptionkey@0"
I20260810 18:41:49.015512 10457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:41:49.054224 10457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:41:49.059190 10457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:41:49.059535 10457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:41:49.080688 10457 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:32859
I20260810 18:41:49.080749 10564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:32859 every 8 connection(s)
I20260810 18:41:49.082352 10457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:41:49.086776 10566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:49.091663 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 10457
I20260810 18:41:49.091925 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:41:49.092643 5011 external_mini_cluster.cc:1428] Setting key d25ef73294bc88a65a956343b0a292a6
I20260810 18:41:49.100193 10566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aaff2299495e42d09ed87a73241e012f: Bootstrap starting.
I20260810 18:41:49.104498 10566 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aaff2299495e42d09ed87a73241e012f: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:49.105723 10566 log.cc:824] T 00000000000000000000000000000000 P aaff2299495e42d09ed87a73241e012f: Log is configured to *not* fsync() on all Append() calls
I20260810 18:41:49.109539 10566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aaff2299495e42d09ed87a73241e012f: No bootstrap required, opened a new log
Aug 10 18:41:49 dist-test-slave-7kzw krb5kdc[10438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:49.122278 10566 raft_consensus.cc:348] T 00000000000000000000000000000000 P aaff2299495e42d09ed87a73241e012f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aaff2299495e42d09ed87a73241e012f" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 32859 } }
I20260810 18:41:49.122933 10566 raft_consensus.cc:374] T 00000000000000000000000000000000 P aaff2299495e42d09ed87a73241e012f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:49.123071 10566 raft_consensus.cc:729] T 00000000000000000000000000000000 P aaff2299495e42d09ed87a73241e012f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaff2299495e42d09ed87a73241e012f, State: Initialized, Role: FOLLOWER
I20260810 18:41:49.123476 10569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:49.095070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56414 (local address 127.4.228.254:32859)
0810 18:41:49.096077 (+ 1007us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:49.096093 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:49.096166 (+ 73us) server_negotiation.cc:407] Connection header received
0810 18:41:49.098818 (+ 2652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:49.098853 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:49.099434 (+ 581us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:49.099973 (+ 539us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:49.101881 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:49.104053 (+ 2172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:49.105584 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:49.106819 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:49.106989 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:49.112185 (+ 5196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:49.112204 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:49.112230 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:49.112296 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:49.117163 (+ 4867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:49.118043 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:49.118054 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:49.118065 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:49.118334 (+ 269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:49.118846 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:49.118851 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:49.118855 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:49.119888 (+ 1033us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:49.120290 (+ 402us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:49.120819 (+ 529us) server_negotiation.cc:299] Negotiation successful
0810 18:41:49.121629 (+ 810us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":704,"thread_start_us":443,"threads_started":1}
I20260810 18:41:49.123914 10566 consensus_queue.cc:260] T 00000000000000000000000000000000 P aaff2299495e42d09ed87a73241e012f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aaff2299495e42d09ed87a73241e012f" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 32859 } }
I20260810 18:41:49.124287 10566 raft_consensus.cc:388] T 00000000000000000000000000000000 P aaff2299495e42d09ed87a73241e012f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:41:49.124454 10566 raft_consensus.cc:482] T 00000000000000000000000000000000 P aaff2299495e42d09ed87a73241e012f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:41:49.124677 10566 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aaff2299495e42d09ed87a73241e012f [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:49.127804 10566 raft_consensus.cc:504] T 00000000000000000000000000000000 P aaff2299495e42d09ed87a73241e012f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aaff2299495e42d09ed87a73241e012f" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 32859 } }
I20260810 18:41:49.128408 10566 leader_election.cc:302] T 00000000000000000000000000000000 P aaff2299495e42d09ed87a73241e012f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aaff2299495e42d09ed87a73241e012f; no voters:
I20260810 18:41:49.129204 10566 leader_election.cc:288] T 00000000000000000000000000000000 P aaff2299495e42d09ed87a73241e012f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:41:49.129393 10571 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aaff2299495e42d09ed87a73241e012f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:41:49.130597 10571 raft_consensus.cc:686] T 00000000000000000000000000000000 P aaff2299495e42d09ed87a73241e012f [term 1 LEADER]: Becoming Leader. State: Replica: aaff2299495e42d09ed87a73241e012f, State: Running, Role: LEADER
I20260810 18:41:49.131240 10571 consensus_queue.cc:237] T 00000000000000000000000000000000 P aaff2299495e42d09ed87a73241e012f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aaff2299495e42d09ed87a73241e012f" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 32859 } }
I20260810 18:41:49.131804 10566 sys_catalog.cc:564] T 00000000000000000000000000000000 P aaff2299495e42d09ed87a73241e012f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:41:49.135525 10573 sys_catalog.cc:455] T 00000000000000000000000000000000 P aaff2299495e42d09ed87a73241e012f [sys.catalog]: SysCatalogTable state changed. Reason: New leader aaff2299495e42d09ed87a73241e012f. Latest consensus state: current_term: 1 leader_uuid: "aaff2299495e42d09ed87a73241e012f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aaff2299495e42d09ed87a73241e012f" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 32859 } } }
I20260810 18:41:49.136108 10573 sys_catalog.cc:458] T 00000000000000000000000000000000 P aaff2299495e42d09ed87a73241e012f [sys.catalog]: This master's current role is: LEADER
I20260810 18:41:49.136720 10572 sys_catalog.cc:455] T 00000000000000000000000000000000 P aaff2299495e42d09ed87a73241e012f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aaff2299495e42d09ed87a73241e012f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aaff2299495e42d09ed87a73241e012f" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 32859 } } }
I20260810 18:41:49.137593 10572 sys_catalog.cc:458] T 00000000000000000000000000000000 P aaff2299495e42d09ed87a73241e012f [sys.catalog]: This master's current role is: LEADER
I20260810 18:41:49.138718 10579 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:41:49.145478 10579 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:41:49.155841 10579 catalog_manager.cc:1347] Generated new cluster ID: f5848935b07547b8b55479b85add446b
I20260810 18:41:49.156014 10579 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:41:49.184157 10579 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:41:49.185649 10579 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:41:49.199249 10579 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aaff2299495e42d09ed87a73241e012f: Generated new TSK 0
I20260810 18:41:49.200453 10579 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.228.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab.
Entry for principal kudu/127.4.228.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab.
I20260810 18:41:49.271456 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.193:0
--local_ip_for_outbound_sockets=127.4.228.193
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:32859
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:40429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:41:49.659909 10599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:41:49.660279 10599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:41:49.660456 10599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:41:49.668803 10599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:41:49.668931 10599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:41:49.669166 10599 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.193
I20260810 18:41:49.679841 10599 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:40429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:32859
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.193
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:41:49.681468 10599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:41:49.683051 10599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:41:49.697475 10605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:49.697479 10608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:49.697458 10606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:49.700344 10599 server_base.cc:1034] running on GCE node
I20260810 18:41:49.701150 10599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:41:49.702455 10599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:41:49.703783 10599 hybrid_clock.cc:648] HybridClock initialized: now 1786387309703706 us; error 66 us; skew 500 ppm
I20260810 18:41:49.704136 10599 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:41:49 dist-test-slave-7kzw krb5kdc[10438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387309, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:41:49.707911 10599 init.cc:377] Logged in from keytab as kudu/127.4.228.193@KRBTEST.COM (short username kudu)
I20260810 18:41:49.709766 10599 webserver.cc:460] Webserver started at http://127.4.228.193:46355/ using document root <none> and password file <none>
I20260810 18:41:49.710669 10599 fs_manager.cc:359] Metadata directory not provided
I20260810 18:41:49.710837 10599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:41:49.711186 10599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:41:49.714416 10599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-0/data/instance:
uuid: "2ad0b8c4f5d44bcd9497f2f192fabd28"
format_stamp: "Formatted at 2026-08-10 18:41:49 on dist-test-slave-7kzw"
server_key: "377efabdd1f6ae9b376da5111f591bd8"
server_key_iv: "f5647585d4ef70e05ccc881a909917ec"
server_key_version: "encryptionkey@0"
I20260810 18:41:49.715340 10599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance:
uuid: "2ad0b8c4f5d44bcd9497f2f192fabd28"
format_stamp: "Formatted at 2026-08-10 18:41:49 on dist-test-slave-7kzw"
server_key: "377efabdd1f6ae9b376da5111f591bd8"
server_key_iv: "f5647585d4ef70e05ccc881a909917ec"
server_key_version: "encryptionkey@0"
I20260810 18:41:49.721369 10599 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 18:41:49.725461 10617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:49.727263 10599 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 18:41:49.727505 10599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-0/wal
uuid: "2ad0b8c4f5d44bcd9497f2f192fabd28"
format_stamp: "Formatted at 2026-08-10 18:41:49 on dist-test-slave-7kzw"
server_key: "377efabdd1f6ae9b376da5111f591bd8"
server_key_iv: "f5647585d4ef70e05ccc881a909917ec"
server_key_version: "encryptionkey@0"
I20260810 18:41:49.727777 10599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:41:49.751008 10599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:41:49.756088 10599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:41:49.756448 10599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:41:49.757836 10599 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:41:49.759470 10599 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:41:49.759589 10599 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:49.759738 10599 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:41:49.759814 10599 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:49.797264 10599 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.193:38777
I20260810 18:41:49.797293 10779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.193:38777 every 8 connection(s)
I20260810 18:41:49.798771 10599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
I20260810 18:41:49.803953 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 10599
I20260810 18:41:49.804234 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance
I20260810 18:41:49.804932 5011 external_mini_cluster.cc:1428] Setting key 1d54d097fbdc84b11d478f3b357331f2
Aug 10 18:41:49 dist-test-slave-7kzw krb5kdc[10438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387309, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:49.822034 10782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:49.801990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:60189 (local address 127.4.228.254:32859)
0810 18:41:49.802694 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:49.802699 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:49.803733 (+ 1034us) server_negotiation.cc:407] Connection header received
0810 18:41:49.806044 (+ 2311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:49.806050 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:49.806159 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:49.806295 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:49.808281 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:49.809015 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:49.810818 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:49.811231 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:49.811379 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:49.815477 (+ 4098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:49.815482 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:49.815486 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:49.815525 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:49.818211 (+ 2686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:49.819158 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:49.819162 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:49.819165 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:49.819324 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:49.819875 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:49.819882 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:49.819884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:49.820276 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:49.820443 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:49.821381 (+ 938us) server_negotiation.cc:299] Negotiation successful
0810 18:41:49.821628 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":320,"threads_started":1}
I20260810 18:41:49.823251 10783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:49.802657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:32859 (local address 127.4.228.193:60189)
0810 18:41:49.803504 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:49.803553 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:49.805687 (+ 2134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:49.806640 (+ 953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:49.806651 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:49.807413 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:49.808074 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:49.808098 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:49.809212 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:49.809216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:49.810654 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:49.810669 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:49.811490 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:49.811500 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:49.811775 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:49.812539 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:41:49.812565 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:49.815177 (+ 2612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:49.818391 (+ 3214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:49.818402 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:49.818424 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:49.819001 (+ 577us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:49.819450 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:49.819456 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:49.819460 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:49.819746 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:49.820417 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:49.820426 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:49.820937 (+ 511us) client_negotiation.cc:769] Sending connection context
0810 18:41:49.821259 (+ 322us) client_negotiation.cc:240] Negotiation successful
0810 18:41:49.821876 (+ 617us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":335,"threads_started":1}
I20260810 18:41:49.825117 10780 heartbeater.cc:347] Connected to a master server at 127.4.228.254:32859
I20260810 18:41:49.825537 10780 heartbeater.cc:464] Registering TS with master...
I20260810 18:41:49.826684 10780 heartbeater.cc:511] Master 127.4.228.254:32859 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.4.228.194@KRBTEST.COM; defaulting to no policy
I20260810 18:41:49.830287 10516 ts_manager.cc:194] Registered new tserver with Master: 2ad0b8c4f5d44bcd9497f2f192fabd28 (127.4.228.193:38777)
I20260810 18:41:49.832355 10516 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.193@KRBTEST.COM'} at 127.4.228.193:60189
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab.
Entry for principal kudu/127.4.228.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab.
I20260810 18:41:49.866446 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.194:0
--local_ip_for_outbound_sockets=127.4.228.194
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:32859
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:40429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:41:50.252627 10788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:41:50.252959 10788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:41:50.253121 10788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:41:50.260676 10788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:41:50.260811 10788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:41:50.260994 10788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.194
I20260810 18:41:50.272395 10788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:40429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:32859
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.194
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:41:50.273945 10788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:41:50.275759 10788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:41:50.291980 10794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:50.291980 10798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:50.291944 10795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:50.293341 10788 server_base.cc:1034] running on GCE node
I20260810 18:41:50.294101 10788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:41:50.295384 10788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:41:50.296624 10788 hybrid_clock.cc:648] HybridClock initialized: now 1786387310296573 us; error 34 us; skew 500 ppm
I20260810 18:41:50.296993 10788 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:41:50 dist-test-slave-7kzw krb5kdc[10438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387310, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:41:50.301841 10788 init.cc:377] Logged in from keytab as kudu/127.4.228.194@KRBTEST.COM (short username kudu)
I20260810 18:41:50.304829 10788 webserver.cc:460] Webserver started at http://127.4.228.194:33749/ using document root <none> and password file <none>
I20260810 18:41:50.305699 10788 fs_manager.cc:359] Metadata directory not provided
I20260810 18:41:50.305879 10788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:41:50.306249 10788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:41:50.310079 10788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-1/data/instance:
uuid: "1d8608ef76504247b27dfc2cba6a9ee4"
format_stamp: "Formatted at 2026-08-10 18:41:50 on dist-test-slave-7kzw"
server_key: "00601b22e72823cedcdae78457b7f10f"
server_key_iv: "d5fade2088aaa94a88ad7bcba2d69702"
server_key_version: "encryptionkey@0"
I20260810 18:41:50.311085 10788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance:
uuid: "1d8608ef76504247b27dfc2cba6a9ee4"
format_stamp: "Formatted at 2026-08-10 18:41:50 on dist-test-slave-7kzw"
server_key: "00601b22e72823cedcdae78457b7f10f"
server_key_iv: "d5fade2088aaa94a88ad7bcba2d69702"
server_key_version: "encryptionkey@0"
I20260810 18:41:50.317728 10788 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 18:41:50.322721 10810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:50.324489 10788 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:41:50.324710 10788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-1/wal
uuid: "1d8608ef76504247b27dfc2cba6a9ee4"
format_stamp: "Formatted at 2026-08-10 18:41:50 on dist-test-slave-7kzw"
server_key: "00601b22e72823cedcdae78457b7f10f"
server_key_iv: "d5fade2088aaa94a88ad7bcba2d69702"
server_key_version: "encryptionkey@0"
I20260810 18:41:50.324945 10788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:41:50.344631 10788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:41:50.350199 10788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:41:50.350589 10788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:41:50.351833 10788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:41:50.353411 10788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:41:50.353490 10788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:50.353632 10788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:41:50.353688 10788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:50.391353 10788 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.194:45469
I20260810 18:41:50.391398 10975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.194:45469 every 8 connection(s)
I20260810 18:41:50.392756 10788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
I20260810 18:41:50.395457 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 10788
I20260810 18:41:50.395610 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance
I20260810 18:41:50.396229 5011 external_mini_cluster.cc:1428] Setting key 2a4a3108cd0209e4f6f0cdae7d9ddb25
Aug 10 18:41:50 dist-test-slave-7kzw krb5kdc[10438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387310, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:50.416637 10981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:50.396129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:42859 (local address 127.4.228.254:32859)
0810 18:41:50.398517 (+ 2388us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:50.398523 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:50.398546 (+ 23us) server_negotiation.cc:407] Connection header received
0810 18:41:50.400019 (+ 1473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:50.400025 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:50.400129 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:50.400252 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:50.401916 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:50.402747 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:50.404429 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:50.404889 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:50.405035 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:50.409232 (+ 4197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:50.409239 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:50.409244 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:50.409297 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:50.412449 (+ 3152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:50.413628 (+ 1179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:50.413632 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:50.413635 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:50.413786 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:50.414411 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:50.414423 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:50.414426 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:50.414853 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:50.415008 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:50.416040 (+ 1032us) server_negotiation.cc:299] Negotiation successful
0810 18:41:50.416355 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2237,"thread_start_us":2086,"threads_started":1}
I20260810 18:41:50.418323 10982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:50.396743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:32859 (local address 127.4.228.194:42859)
0810 18:41:50.397670 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:50.397733 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:50.399713 (+ 1980us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:50.400532 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:50.400543 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:50.401096 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:50.401722 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:50.401741 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:50.402930 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:50.402934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:50.404251 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:50.404264 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:50.405151 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:50.405160 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:50.405429 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:50.406208 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:41:50.406234 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:50.408918 (+ 2684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:50.412653 (+ 3735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:50.412668 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:50.412692 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:50.413435 (+ 743us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:50.413918 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:50.413926 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:50.413932 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:50.414245 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:50.415007 (+ 762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:50.415019 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:50.415556 (+ 537us) client_negotiation.cc:769] Sending connection context
0810 18:41:50.415920 (+ 364us) client_negotiation.cc:240] Negotiation successful
0810 18:41:50.416681 (+ 761us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":365,"threads_started":1}
I20260810 18:41:50.420365 10976 heartbeater.cc:347] Connected to a master server at 127.4.228.254:32859
I20260810 18:41:50.420939 10976 heartbeater.cc:464] Registering TS with master...
I20260810 18:41:50.422102 10976 heartbeater.cc:511] Master 127.4.228.254:32859 requested a full tablet report, sending...
I20260810 18:41:50.424292 10516 ts_manager.cc:194] Registered new tserver with Master: 1d8608ef76504247b27dfc2cba6a9ee4 (127.4.228.194:45469)
WARNING: no policy specified for kudu/127.4.228.195@KRBTEST.COM; defaulting to no policy
I20260810 18:41:50.425693 10516 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.194@KRBTEST.COM'} at 127.4.228.194:42859
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab.
Entry for principal kudu/127.4.228.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab.
I20260810 18:41:50.463936 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.195:0
--local_ip_for_outbound_sockets=127.4.228.195
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:32859
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:40429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:41:50.836494 10780 heartbeater.cc:503] Master 127.4.228.254:32859 was elected leader, sending a full tablet report...
W20260810 18:41:50.852205 10985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:41:50.852641 10985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:41:50.852856 10985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:41:50.861616 10985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:41:50.861815 10985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:41:50.862032 10985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.195
I20260810 18:41:50.873554 10985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:40429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:32859
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.195
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:41:50.875645 10985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:41:50.877502 10985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:41:50.893051 10997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:50.893064 10999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:50.893044 10993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:50.894634 10985 server_base.cc:1034] running on GCE node
I20260810 18:41:50.895437 10985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:41:50.896592 10985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:41:50.897871 10985 hybrid_clock.cc:648] HybridClock initialized: now 1786387310897783 us; error 78 us; skew 500 ppm
I20260810 18:41:50.898277 10985 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:41:50 dist-test-slave-7kzw krb5kdc[10438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387310, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:41:50.902338 10985 init.cc:377] Logged in from keytab as kudu/127.4.228.195@KRBTEST.COM (short username kudu)
I20260810 18:41:50.904273 10985 webserver.cc:460] Webserver started at http://127.4.228.195:39685/ using document root <none> and password file <none>
I20260810 18:41:50.905090 10985 fs_manager.cc:359] Metadata directory not provided
I20260810 18:41:50.905260 10985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:41:50.905680 10985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:41:50.909139 10985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-2/data/instance:
uuid: "285aa627e0e94a808122df94df4bbbef"
format_stamp: "Formatted at 2026-08-10 18:41:50 on dist-test-slave-7kzw"
server_key: "dade8999ce05c837f30e01512b7d80ff"
server_key_iv: "91a6bf70313c71d931ddeb22cb59c6f0"
server_key_version: "encryptionkey@0"
I20260810 18:41:50.910073 10985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance:
uuid: "285aa627e0e94a808122df94df4bbbef"
format_stamp: "Formatted at 2026-08-10 18:41:50 on dist-test-slave-7kzw"
server_key: "dade8999ce05c837f30e01512b7d80ff"
server_key_iv: "91a6bf70313c71d931ddeb22cb59c6f0"
server_key_version: "encryptionkey@0"
I20260810 18:41:50.916558 10985 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.003s
I20260810 18:41:50.921777 11013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:50.923614 10985 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260810 18:41:50.923866 10985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-2/wal
uuid: "285aa627e0e94a808122df94df4bbbef"
format_stamp: "Formatted at 2026-08-10 18:41:50 on dist-test-slave-7kzw"
server_key: "dade8999ce05c837f30e01512b7d80ff"
server_key_iv: "91a6bf70313c71d931ddeb22cb59c6f0"
server_key_version: "encryptionkey@0"
I20260810 18:41:50.924134 10985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:41:50.951653 10985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:41:50.956866 10985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:41:50.957216 10985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:41:50.958400 10985 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:41:50.960011 10985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:41:50.960134 10985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:50.960297 10985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:41:50.960374 10985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:50.999341 10985 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.195:35335
I20260810 18:41:50.999655 11199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.195:35335 every 8 connection(s)
I20260810 18:41:51.000917 10985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
I20260810 18:41:51.005551 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 10985
I20260810 18:41:51.005780 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance
I20260810 18:41:51.006296 5011 external_mini_cluster.cc:1428] Setting key f0f4a3b3e42fe21dd9242b7b0157aad5
Aug 10 18:41:51 dist-test-slave-7kzw krb5kdc[10438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387310, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:51.025744 11203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:51.003975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:57499 (local address 127.4.228.254:32859)
0810 18:41:51.004486 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:51.004491 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:51.005618 (+ 1127us) server_negotiation.cc:407] Connection header received
0810 18:41:51.007903 (+ 2285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:51.007908 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:51.008013 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:51.008143 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:51.009906 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:51.010788 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:51.012442 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:51.012859 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:51.013000 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:51.017718 (+ 4718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:51.017725 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:51.017729 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:51.017772 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:51.021510 (+ 3738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:51.022746 (+ 1236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:51.022755 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:51.022762 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:51.022930 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:51.023525 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:51.023529 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:51.023531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:51.023877 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:51.024077 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:51.024969 (+ 892us) server_negotiation.cc:299] Negotiation successful
0810 18:41:51.025264 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":249,"threads_started":1}
I20260810 18:41:51.027081 11205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:51.004536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:32859 (local address 127.4.228.195:57499)
0810 18:41:51.005411 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:51.005469 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:51.007559 (+ 2090us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:51.008436 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:51.008445 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:51.009039 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:51.009699 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:51.009719 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:51.010959 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:51.010973 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:51.012272 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:51.012286 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:51.013127 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:51.013135 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:51.013386 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:51.014177 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:41:51.014209 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:51.017320 (+ 3111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:51.021867 (+ 4547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:51.021882 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:51.021912 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:51.022598 (+ 686us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:51.023074 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:51.023079 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:51.023084 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:51.023363 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:51.024048 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:51.024058 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:51.024516 (+ 458us) client_negotiation.cc:769] Sending connection context
0810 18:41:51.024859 (+ 343us) client_negotiation.cc:240] Negotiation successful
0810 18:41:51.025538 (+ 679us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":370,"threads_started":1}
I20260810 18:41:51.029014 11201 heartbeater.cc:347] Connected to a master server at 127.4.228.254:32859
I20260810 18:41:51.029451 11201 heartbeater.cc:464] Registering TS with master...
I20260810 18:41:51.030541 11201 heartbeater.cc:511] Master 127.4.228.254:32859 requested a full tablet report, sending...
I20260810 18:41:51.032567 10516 ts_manager.cc:194] Registered new tserver with Master: 285aa627e0e94a808122df94df4bbbef (127.4.228.195:35335)
I20260810 18:41:51.033697 10516 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.195@KRBTEST.COM'} at 127.4.228.195:57499
I20260810 18:41:51.036399 5011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 18:41:51 dist-test-slave-7kzw krb5kdc[10438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 18:41:51 dist-test-slave-7kzw krb5kdc[10438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.193@KRBTEST.COM
I20260810 18:41:51.105829 11212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:51.085776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54634 (local address 127.4.228.193:38777)
0810 18:41:51.087746 (+ 1970us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:51.087758 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:51.087803 (+ 45us) server_negotiation.cc:407] Connection header received
0810 18:41:51.087897 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:51.087908 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:51.088203 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:51.088380 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:51.089934 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:51.091133 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:51.092718 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:51.093363 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:51.093496 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:51.097856 (+ 4360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:51.097872 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:51.097890 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:51.097934 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:51.101500 (+ 3566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:51.103003 (+ 1503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:51.103012 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:51.103015 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:51.103182 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:51.103747 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:51.103751 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:51.103753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:51.104200 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:51.104476 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:51.105055 (+ 579us) server_negotiation.cc:299] Negotiation successful
0810 18:41:51.105354 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1757,"thread_start_us":1583,"threads_started":1}
Aug 10 18:41:51 dist-test-slave-7kzw krb5kdc[10438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:51.135324 11203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:51.112256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56424 (local address 127.4.228.254:32859)
0810 18:41:51.112562 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:51.112573 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:51.112912 (+ 339us) server_negotiation.cc:407] Connection header received
0810 18:41:51.113171 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:51.113176 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:51.113303 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:51.113413 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:51.115053 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:51.115938 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:51.117771 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:51.118246 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:51.118375 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:51.128350 (+ 9975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:51.128357 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:51.128361 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:51.128410 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:51.131827 (+ 3417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:51.132755 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:51.132762 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:51.132767 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:51.132944 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:51.133456 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:51.133462 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:51.133466 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:51.133888 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:51.134150 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:51.134649 (+ 499us) server_negotiation.cc:299] Negotiation successful
0810 18:41:51.134863 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
W20260810 18:41:51.136082 10515 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56424
I20260810 18:41:51.137477 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 10599
I20260810 18:41:51.152988 11212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:51.138764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54640 (local address 127.4.228.193:38777)
0810 18:41:51.138978 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:51.138987 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:51.139013 (+ 26us) server_negotiation.cc:407] Connection header received
0810 18:41:51.139416 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:51.139421 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:51.139525 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:51.139841 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:51.141175 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:51.141948 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:51.143386 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:51.143884 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:51.143998 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:51.146708 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:51.146713 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:51.146717 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:51.146784 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:51.149736 (+ 2952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:51.150679 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:51.150684 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:51.150686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:51.150850 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:51.151357 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:51.151362 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:51.151364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:51.151663 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:51.151968 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:51.152382 (+ 414us) server_negotiation.cc:299] Negotiation successful
0810 18:41:51.152626 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 18:41:51.153914 10773 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:41:51.242648 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 10599
I20260810 18:41:51.267134 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 10788
Aug 10 18:41:51 dist-test-slave-7kzw krb5kdc[10438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.194@KRBTEST.COM
I20260810 18:41:51.286602 11221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:51.268195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48680 (local address 127.4.228.194:45469)
0810 18:41:51.268794 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:51.268801 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:51.268830 (+ 29us) server_negotiation.cc:407] Connection header received
0810 18:41:51.268899 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:51.268906 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:51.269133 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:51.269306 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:51.270695 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:51.271740 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:51.273125 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:51.273822 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:51.273959 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:51.279259 (+ 5300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:51.279277 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:51.279295 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:51.279340 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:51.283249 (+ 3909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:51.284148 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:51.284158 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:51.284161 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:51.284317 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:51.284753 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:51.284758 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:51.284763 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:51.285163 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:51.285511 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:51.285851 (+ 340us) server_negotiation.cc:299] Negotiation successful
0810 18:41:51.286146 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":312,"threads_started":1}
I20260810 18:41:51.288292 10971 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:41:51.389565 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 10788
I20260810 18:41:51.421237 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 10985
Aug 10 18:41:51 dist-test-slave-7kzw krb5kdc[10438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.195@KRBTEST.COM
I20260810 18:41:51.440835 11224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:51.422450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51450 (local address 127.4.228.195:35335)
0810 18:41:51.423160 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:51.423169 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:51.423202 (+ 33us) server_negotiation.cc:407] Connection header received
0810 18:41:51.423280 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:51.423288 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:51.423520 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:51.423662 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:51.424949 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:51.425975 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:51.427595 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:51.428489 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:51.428632 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:51.433912 (+ 5280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:51.433927 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:51.433946 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:51.433996 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:51.437190 (+ 3194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:51.438031 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:51.438039 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:51.438042 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:51.438187 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:51.438663 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:51.438667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:51.438669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:51.439151 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:51.439499 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:51.439892 (+ 393us) server_negotiation.cc:299] Negotiation successful
0810 18:41:51.440195 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":343,"threads_started":1}
I20260810 18:41:51.442613 11194 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:41:51.529922 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 10985
I20260810 18:41:51.556011 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 10457
I20260810 18:41:51.557502 10560 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:41:51.668032 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 10457
2026-08-10T18:41:51Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4722 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:41:51 dist-test-slave-7kzw krb5kdc[11232](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:41:51 dist-test-slave-7kzw krb5kdc[11232](info): set up 2 sockets
Aug 10 18:41:51 dist-test-slave-7kzw krb5kdc[11232](info): commencing operation
krb5kdc: starting...
W20260810 18:41:53.019237 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.296s user 0.003s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:41:53 dist-test-slave-7kzw krb5kdc[11232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387313, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:41:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:41:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.228.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
I20260810 18:41:53.296130 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:38533
--webserver_interface=127.4.228.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:40531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:38533
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:41:53.663232 11252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:41:53.663731 11252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:41:53.663930 11252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:41:53.673540 11252 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:41:53.673669 11252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:41:53.673718 11252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:41:53.673770 11252 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:41:53.673820 11252 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:41:53.684441 11252 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:40531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:38533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:38533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:41:53.685896 11252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:41:53.687431 11252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:41:53.700726 11259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:53.700608 11263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:53.700692 11261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:53.701612 11252 server_base.cc:1034] running on GCE node
I20260810 18:41:53.702736 11252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:41:53.704423 11252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:41:53.705720 11252 hybrid_clock.cc:648] HybridClock initialized: now 1786387313705631 us; error 35 us; skew 500 ppm
I20260810 18:41:53.706182 11252 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:41:53 dist-test-slave-7kzw krb5kdc[11232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387313, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:41:53.710155 11252 init.cc:377] Logged in from keytab as kudu/127.4.228.254@KRBTEST.COM (short username kudu)
I20260810 18:41:53.711742 11252 fs_manager.cc:359] Metadata directory not provided
I20260810 18:41:53.711911 11252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:41:53.712293 11252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:41:53.715898 11252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "3fa02cb275da4ab3b78cd49f39284d53"
format_stamp: "Formatted at 2026-08-10 18:41:53 on dist-test-slave-7kzw"
server_key: "4ebe8e3e82a01ffd259a96ac41c976f6"
server_key_iv: "b1dca33a250bcf94cd11baa2f2b3220b"
server_key_version: "encryptionkey@0"
I20260810 18:41:53.716851 11252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "3fa02cb275da4ab3b78cd49f39284d53"
format_stamp: "Formatted at 2026-08-10 18:41:53 on dist-test-slave-7kzw"
server_key: "4ebe8e3e82a01ffd259a96ac41c976f6"
server_key_iv: "b1dca33a250bcf94cd11baa2f2b3220b"
server_key_version: "encryptionkey@0"
I20260810 18:41:53.723037 11252 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260810 18:41:53.726899 11273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:53.728484 11252 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:41:53.728719 11252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "3fa02cb275da4ab3b78cd49f39284d53"
format_stamp: "Formatted at 2026-08-10 18:41:53 on dist-test-slave-7kzw"
server_key: "4ebe8e3e82a01ffd259a96ac41c976f6"
server_key_iv: "b1dca33a250bcf94cd11baa2f2b3220b"
server_key_version: "encryptionkey@0"
I20260810 18:41:53.728974 11252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:41:53.745415 11252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:41:53.750401 11252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:41:53.750785 11252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:41:53.771858 11252 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:38533
I20260810 18:41:53.771881 11357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:38533 every 8 connection(s)
I20260810 18:41:53.772919 11252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:41:53.774683 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 11252
I20260810 18:41:53.774935 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:41:53.775900 5011 external_mini_cluster.cc:1428] Setting key 6494a414a88a35d70fb0bc866be35cdc
I20260810 18:41:53.778839 11358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 18:41:53 dist-test-slave-7kzw krb5kdc[11232](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387313, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:53.795362 11358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fa02cb275da4ab3b78cd49f39284d53: Bootstrap starting.
I20260810 18:41:53.799337 11358 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3fa02cb275da4ab3b78cd49f39284d53: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:53.800664 11358 log.cc:824] T 00000000000000000000000000000000 P 3fa02cb275da4ab3b78cd49f39284d53: Log is configured to *not* fsync() on all Append() calls
I20260810 18:41:53.804725 11358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fa02cb275da4ab3b78cd49f39284d53: No bootstrap required, opened a new log
I20260810 18:41:53.804853 11361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:53.778341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50350 (local address 127.4.228.254:38533)
0810 18:41:53.779312 (+ 971us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:53.779325 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:53.779370 (+ 45us) server_negotiation.cc:407] Connection header received
0810 18:41:53.781047 (+ 1677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:53.781070 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:53.781537 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:53.782060 (+ 523us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:53.784972 (+ 2912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:53.786668 (+ 1696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:53.788765 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:53.789606 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:53.789826 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:53.795585 (+ 5759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:53.795602 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:53.795633 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:53.795690 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:53.799364 (+ 3674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:53.800254 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:53.800263 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:53.800271 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:53.800477 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:53.800995 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:53.801001 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:53.801005 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:53.801876 (+ 871us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:53.802142 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:53.802821 (+ 679us) server_negotiation.cc:299] Negotiation successful
0810 18:41:53.803343 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":728,"thread_start_us":439,"threads_started":1}
I20260810 18:41:53.813436 11358 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3fa02cb275da4ab3b78cd49f39284d53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fa02cb275da4ab3b78cd49f39284d53" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 38533 } }
I20260810 18:41:53.813869 11358 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3fa02cb275da4ab3b78cd49f39284d53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:53.814018 11358 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3fa02cb275da4ab3b78cd49f39284d53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fa02cb275da4ab3b78cd49f39284d53, State: Initialized, Role: FOLLOWER
I20260810 18:41:53.814844 11358 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3fa02cb275da4ab3b78cd49f39284d53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fa02cb275da4ab3b78cd49f39284d53" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 38533 } }
I20260810 18:41:53.815158 11358 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3fa02cb275da4ab3b78cd49f39284d53 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:41:53.815333 11358 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3fa02cb275da4ab3b78cd49f39284d53 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:41:53.815559 11358 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3fa02cb275da4ab3b78cd49f39284d53 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:53.818053 11358 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3fa02cb275da4ab3b78cd49f39284d53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fa02cb275da4ab3b78cd49f39284d53" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 38533 } }
I20260810 18:41:53.818742 11358 leader_election.cc:302] T 00000000000000000000000000000000 P 3fa02cb275da4ab3b78cd49f39284d53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3fa02cb275da4ab3b78cd49f39284d53; no voters:
I20260810 18:41:53.819463 11358 leader_election.cc:288] T 00000000000000000000000000000000 P 3fa02cb275da4ab3b78cd49f39284d53 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:41:53.819597 11366 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3fa02cb275da4ab3b78cd49f39284d53 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:41:53.820978 11366 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3fa02cb275da4ab3b78cd49f39284d53 [term 1 LEADER]: Becoming Leader. State: Replica: 3fa02cb275da4ab3b78cd49f39284d53, State: Running, Role: LEADER
I20260810 18:41:53.821523 11366 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3fa02cb275da4ab3b78cd49f39284d53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fa02cb275da4ab3b78cd49f39284d53" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 38533 } }
I20260810 18:41:53.822113 11358 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3fa02cb275da4ab3b78cd49f39284d53 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:41:53.825373 11368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fa02cb275da4ab3b78cd49f39284d53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3fa02cb275da4ab3b78cd49f39284d53. Latest consensus state: current_term: 1 leader_uuid: "3fa02cb275da4ab3b78cd49f39284d53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fa02cb275da4ab3b78cd49f39284d53" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 38533 } } }
I20260810 18:41:53.825870 11368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fa02cb275da4ab3b78cd49f39284d53 [sys.catalog]: This master's current role is: LEADER
I20260810 18:41:53.825472 11367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fa02cb275da4ab3b78cd49f39284d53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3fa02cb275da4ab3b78cd49f39284d53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fa02cb275da4ab3b78cd49f39284d53" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 38533 } } }
I20260810 18:41:53.826365 11367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fa02cb275da4ab3b78cd49f39284d53 [sys.catalog]: This master's current role is: LEADER
I20260810 18:41:53.827773 11377 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:41:53.833833 11377 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:41:53.844446 11377 catalog_manager.cc:1347] Generated new cluster ID: c7591dbc80d043c182e51acc85ca31cc
I20260810 18:41:53.844624 11377 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:41:53.855185 11377 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:41:53.857303 11377 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:41:53.885787 11377 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3fa02cb275da4ab3b78cd49f39284d53: Generated new TSK 0
I20260810 18:41:53.887022 11377 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.228.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab.
Entry for principal kudu/127.4.228.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab.
I20260810 18:41:53.968308 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.193:0
--local_ip_for_outbound_sockets=127.4.228.193
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:38533
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:40531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:41:54.364674 11391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:41:54.365073 11391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:41:54.365259 11391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:41:54.372727 11391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:41:54.372916 11391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:41:54.373122 11391 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.193
I20260810 18:41:54.383394 11391 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:40531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:38533
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.193
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:41:54.384994 11391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:41:54.386721 11391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:41:54.401189 11399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:54.401201 11400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:54.401198 11402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:54.402629 11391 server_base.cc:1034] running on GCE node
I20260810 18:41:54.403383 11391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:41:54.404551 11391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:41:54.405834 11391 hybrid_clock.cc:648] HybridClock initialized: now 1786387314405792 us; error 36 us; skew 500 ppm
I20260810 18:41:54.406225 11391 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:41:54 dist-test-slave-7kzw krb5kdc[11232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387314, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:41:54.409955 11391 init.cc:377] Logged in from keytab as kudu/127.4.228.193@KRBTEST.COM (short username kudu)
I20260810 18:41:54.411458 11391 fs_manager.cc:359] Metadata directory not provided
I20260810 18:41:54.411609 11391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:41:54.411981 11391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:41:54.415226 11391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-0/data/instance:
uuid: "c2a07565bdf645cbbe11a58f1a2b1bfe"
format_stamp: "Formatted at 2026-08-10 18:41:54 on dist-test-slave-7kzw"
server_key: "0b19c3b4e0734a73929dff267e8310f3"
server_key_iv: "59c07bd1ce8e7be17a7b8075a09bcb89"
server_key_version: "encryptionkey@0"
I20260810 18:41:54.416103 11391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance:
uuid: "c2a07565bdf645cbbe11a58f1a2b1bfe"
format_stamp: "Formatted at 2026-08-10 18:41:54 on dist-test-slave-7kzw"
server_key: "0b19c3b4e0734a73929dff267e8310f3"
server_key_iv: "59c07bd1ce8e7be17a7b8075a09bcb89"
server_key_version: "encryptionkey@0"
I20260810 18:41:54.421835 11391 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260810 18:41:54.425987 11411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:54.427752 11391 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:41:54.427961 11391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-0/wal
uuid: "c2a07565bdf645cbbe11a58f1a2b1bfe"
format_stamp: "Formatted at 2026-08-10 18:41:54 on dist-test-slave-7kzw"
server_key: "0b19c3b4e0734a73929dff267e8310f3"
server_key_iv: "59c07bd1ce8e7be17a7b8075a09bcb89"
server_key_version: "encryptionkey@0"
I20260810 18:41:54.428220 11391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:41:54.446597 11391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:41:54.451788 11391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:41:54.452137 11391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:41:54.453330 11391 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:41:54.455000 11391 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:41:54.455140 11391 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:54.455298 11391 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:41:54.455405 11391 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:54.494882 11391 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.193:35075
I20260810 18:41:54.494921 11593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.193:35075 every 8 connection(s)
I20260810 18:41:54.496387 11391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
I20260810 18:41:54.500116 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 11391
I20260810 18:41:54.500394 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance
I20260810 18:41:54.501364 5011 external_mini_cluster.cc:1428] Setting key 2133e99eca596059b8b7d50c54a93ad9
Aug 10 18:41:54 dist-test-slave-7kzw krb5kdc[11232](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387314, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:54.522535 11597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:54.499827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:56643 (local address 127.4.228.254:38533)
0810 18:41:54.500410 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:54.500415 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:54.501994 (+ 1579us) server_negotiation.cc:407] Connection header received
0810 18:41:54.504390 (+ 2396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:54.504397 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:54.504505 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:54.504666 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:54.506455 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:54.507337 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:54.509137 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:54.509593 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:54.509762 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:54.514046 (+ 4284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:54.514052 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:54.514056 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:54.514098 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:54.517698 (+ 3600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:54.519032 (+ 1334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:54.519038 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:54.519041 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:54.519206 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:54.520076 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:54.520081 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:54.520083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:54.520441 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:54.520587 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:54.521883 (+ 1296us) server_negotiation.cc:299] Negotiation successful
0810 18:41:54.522177 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":317,"threads_started":1}
I20260810 18:41:54.524593 11598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:54.500778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:38533 (local address 127.4.228.193:56643)
0810 18:41:54.501787 (+ 1009us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:54.501847 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:54.504071 (+ 2224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:54.504937 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:54.504946 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:54.505560 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:54.506255 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:54.506282 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:54.507523 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:54.507528 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:54.508963 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:54.508978 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:54.509892 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:54.509902 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:54.510171 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:54.511047 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:41:54.511075 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:54.513731 (+ 2656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:54.517915 (+ 4184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:54.517931 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:54.517957 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:54.518833 (+ 876us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:54.519447 (+ 614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:54.519456 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:54.519464 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:54.519891 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:54.520631 (+ 740us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:54.520654 (+ 23us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:54.521252 (+ 598us) client_negotiation.cc:769] Sending connection context
0810 18:41:54.521678 (+ 426us) client_negotiation.cc:240] Negotiation successful
0810 18:41:54.522449 (+ 771us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"thread_start_us":413,"threads_started":1}
I20260810 18:41:54.526768 11594 heartbeater.cc:347] Connected to a master server at 127.4.228.254:38533
I20260810 18:41:54.527137 11594 heartbeater.cc:464] Registering TS with master...
I20260810 18:41:54.527984 11594 heartbeater.cc:511] Master 127.4.228.254:38533 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.4.228.194@KRBTEST.COM; defaulting to no policy
I20260810 18:41:54.530869 11297 ts_manager.cc:194] Registered new tserver with Master: c2a07565bdf645cbbe11a58f1a2b1bfe (127.4.228.193:35075)
I20260810 18:41:54.532861 11297 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.193@KRBTEST.COM'} at 127.4.228.193:56643
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab.
Entry for principal kudu/127.4.228.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab.
I20260810 18:41:54.566684 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.194:0
--local_ip_for_outbound_sockets=127.4.228.194
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:38533
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:40531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:41:54.941886 11601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:41:54.942257 11601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:41:54.942437 11601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:41:54.950016 11601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:41:54.950214 11601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:41:54.950414 11601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.194
I20260810 18:41:54.960809 11601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:40531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:38533
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.194
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:41:54.962342 11601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:41:54.963881 11601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:41:54.977876 11608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:54.977881 11607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:54.978629 11610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:54.978925 11601 server_base.cc:1034] running on GCE node
I20260810 18:41:54.979669 11601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:41:54.980919 11601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:41:54.982190 11601 hybrid_clock.cc:648] HybridClock initialized: now 1786387314982115 us; error 68 us; skew 500 ppm
I20260810 18:41:54.982653 11601 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:41:54 dist-test-slave-7kzw krb5kdc[11232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387314, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:41:54.986549 11601 init.cc:377] Logged in from keytab as kudu/127.4.228.194@KRBTEST.COM (short username kudu)
I20260810 18:41:54.987885 11601 fs_manager.cc:359] Metadata directory not provided
I20260810 18:41:54.987993 11601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:41:54.988301 11601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:41:54.991812 11601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-1/data/instance:
uuid: "3024406d59d74006a2d022053a7ba739"
format_stamp: "Formatted at 2026-08-10 18:41:54 on dist-test-slave-7kzw"
server_key: "4b36d64c2e85a08e3526553950f6bff4"
server_key_iv: "389054a33a88e282fb00c7bc0af0a075"
server_key_version: "encryptionkey@0"
I20260810 18:41:54.992733 11601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance:
uuid: "3024406d59d74006a2d022053a7ba739"
format_stamp: "Formatted at 2026-08-10 18:41:54 on dist-test-slave-7kzw"
server_key: "4b36d64c2e85a08e3526553950f6bff4"
server_key_iv: "389054a33a88e282fb00c7bc0af0a075"
server_key_version: "encryptionkey@0"
I20260810 18:41:54.998797 11601 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260810 18:41:55.003103 11616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:55.005138 11601 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:41:55.005322 11601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-1/wal
uuid: "3024406d59d74006a2d022053a7ba739"
format_stamp: "Formatted at 2026-08-10 18:41:54 on dist-test-slave-7kzw"
server_key: "4b36d64c2e85a08e3526553950f6bff4"
server_key_iv: "389054a33a88e282fb00c7bc0af0a075"
server_key_version: "encryptionkey@0"
I20260810 18:41:55.005541 11601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:41:55.051013 11601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:41:55.055981 11601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:41:55.056258 11601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:41:55.057276 11601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:41:55.058743 11601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:41:55.058833 11601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:55.058910 11601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:41:55.058966 11601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:55.095465 11601 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.194:42555
I20260810 18:41:55.095520 11789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.194:42555 every 8 connection(s)
I20260810 18:41:55.096493 11601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
I20260810 18:41:55.106747 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 11601
I20260810 18:41:55.106950 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance
I20260810 18:41:55.107461 5011 external_mini_cluster.cc:1428] Setting key 611cfc6604af8aa41f0c7f137adc95de
Aug 10 18:41:55 dist-test-slave-7kzw krb5kdc[11232](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387314, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:55.121349 11792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:55.099678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:55129 (local address 127.4.228.254:38533)
0810 18:41:55.100185 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:55.100190 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:55.101327 (+ 1137us) server_negotiation.cc:407] Connection header received
0810 18:41:55.103473 (+ 2146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:55.103479 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:55.103589 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:55.103730 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:55.105472 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:55.106300 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:55.108398 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:55.109055 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:55.109172 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:55.113580 (+ 4408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:55.113591 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:55.113599 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:55.113652 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:55.117563 (+ 3911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:55.118440 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:55.118448 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:55.118454 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:55.118661 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:55.119140 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:55.119148 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:55.119154 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:55.119593 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:55.119737 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:55.120860 (+ 1123us) server_negotiation.cc:299] Negotiation successful
0810 18:41:55.121050 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":241,"threads_started":1}
I20260810 18:41:55.123416 11794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:55.100268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:38533 (local address 127.4.228.194:55129)
0810 18:41:55.101135 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:55.101190 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:55.103175 (+ 1985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:55.103998 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:55.104008 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:55.104640 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:55.105269 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:55.105289 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:55.106514 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:55.106520 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:55.108221 (+ 1701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:55.108242 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:55.109344 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:55.109352 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:55.109641 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:55.110577 (+ 936us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:41:55.110607 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:55.113270 (+ 2663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:55.117780 (+ 4510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:55.117789 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:55.117805 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:55.118287 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:55.118806 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:55.118810 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:55.118812 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:55.119009 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:55.119776 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:55.119790 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:55.120365 (+ 575us) client_negotiation.cc:769] Sending connection context
0810 18:41:55.120746 (+ 381us) client_negotiation.cc:240] Negotiation successful
0810 18:41:55.121457 (+ 711us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":364,"threads_started":1}
I20260810 18:41:55.125375 11790 heartbeater.cc:347] Connected to a master server at 127.4.228.254:38533
I20260810 18:41:55.125830 11790 heartbeater.cc:464] Registering TS with master...
I20260810 18:41:55.126713 11790 heartbeater.cc:511] Master 127.4.228.254:38533 requested a full tablet report, sending...
I20260810 18:41:55.128531 11297 ts_manager.cc:194] Registered new tserver with Master: 3024406d59d74006a2d022053a7ba739 (127.4.228.194:42555)
I20260810 18:41:55.129984 11297 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.194@KRBTEST.COM'} at 127.4.228.194:55129
WARNING: no policy specified for kudu/127.4.228.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab.
Entry for principal kudu/127.4.228.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab.
I20260810 18:41:55.175438 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.195:0
--local_ip_for_outbound_sockets=127.4.228.195
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:38533
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:40531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:41:55.536795 11594 heartbeater.cc:503] Master 127.4.228.254:38533 was elected leader, sending a full tablet report...
W20260810 18:41:55.538676 11801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:41:55.539113 11801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:41:55.539292 11801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:41:55.546869 11801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:41:55.547029 11801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:41:55.547227 11801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.195
I20260810 18:41:55.558100 11801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:40531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:38533
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.195
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:41:55.559684 11801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:41:55.561271 11801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:41:55.575575 11807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:55.575615 11809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:55.575655 11811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:55.576694 11801 server_base.cc:1034] running on GCE node
I20260810 18:41:55.577399 11801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:41:55.578559 11801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:41:55.579828 11801 hybrid_clock.cc:648] HybridClock initialized: now 1786387315579753 us; error 67 us; skew 500 ppm
I20260810 18:41:55.580197 11801 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:41:55 dist-test-slave-7kzw krb5kdc[11232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387315, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:41:55.583811 11801 init.cc:377] Logged in from keytab as kudu/127.4.228.195@KRBTEST.COM (short username kudu)
I20260810 18:41:55.585176 11801 fs_manager.cc:359] Metadata directory not provided
I20260810 18:41:55.585279 11801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:41:55.585563 11801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:41:55.588842 11801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-2/data/instance:
uuid: "09600514071b496db42deb2c2704ec76"
format_stamp: "Formatted at 2026-08-10 18:41:55 on dist-test-slave-7kzw"
server_key: "20b1a1a7a6b97293d72c6487cc04a03c"
server_key_iv: "bc307c666e69b75973320c6ec0c977eb"
server_key_version: "encryptionkey@0"
I20260810 18:41:55.589716 11801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance:
uuid: "09600514071b496db42deb2c2704ec76"
format_stamp: "Formatted at 2026-08-10 18:41:55 on dist-test-slave-7kzw"
server_key: "20b1a1a7a6b97293d72c6487cc04a03c"
server_key_iv: "bc307c666e69b75973320c6ec0c977eb"
server_key_version: "encryptionkey@0"
I20260810 18:41:55.595574 11801 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260810 18:41:55.599959 11826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:55.601642 11801 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260810 18:41:55.601876 11801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-2/wal
uuid: "09600514071b496db42deb2c2704ec76"
format_stamp: "Formatted at 2026-08-10 18:41:55 on dist-test-slave-7kzw"
server_key: "20b1a1a7a6b97293d72c6487cc04a03c"
server_key_iv: "bc307c666e69b75973320c6ec0c977eb"
server_key_version: "encryptionkey@0"
I20260810 18:41:55.602161 11801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:41:55.617146 11801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:41:55.622041 11801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:41:55.622388 11801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:41:55.623682 11801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:41:55.625263 11801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:41:55.625355 11801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:55.625511 11801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:41:55.625592 11801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:55.671460 11801 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.195:43733
I20260810 18:41:55.671536 11988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.195:43733 every 8 connection(s)
I20260810 18:41:55.672785 11801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
I20260810 18:41:55.674081 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 11801
I20260810 18:41:55.674302 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance
I20260810 18:41:55.674996 5011 external_mini_cluster.cc:1428] Setting key 0a9b8b8d8c9358b9fd064eade62e8a16
Aug 10 18:41:55 dist-test-slave-7kzw krb5kdc[11232](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387315, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:41:55.698407 11994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:55.678390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:42399 (local address 127.4.228.254:38533)
0810 18:41:55.678936 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:55.678944 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:55.678967 (+ 23us) server_negotiation.cc:407] Connection header received
0810 18:41:55.681142 (+ 2175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:55.681146 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:55.681231 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:55.681388 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:55.683773 (+ 2385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:55.684655 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:55.686435 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:55.686965 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:55.687121 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:55.691611 (+ 4490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:55.691619 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:55.691623 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:55.691667 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:55.694677 (+ 3010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:55.695648 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:55.695653 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:55.695656 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:55.695840 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:55.696436 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:55.696448 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:55.696451 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:55.696792 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:55.696938 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:55.697879 (+ 941us) server_negotiation.cc:299] Negotiation successful
0810 18:41:55.698079 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":1079,"threads_started":1}
I20260810 18:41:55.699677 11991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:55.677699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:38533 (local address 127.4.228.195:42399)
0810 18:41:55.678779 (+ 1080us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:55.678846 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:55.680842 (+ 1996us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:55.681702 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:55.681727 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:55.682632 (+ 905us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:55.683525 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:55.683552 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:55.684839 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:55.684846 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:55.686263 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:55.686277 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:55.687253 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:55.687262 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:55.687552 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:55.688330 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:41:55.688363 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:55.691333 (+ 2970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:55.694866 (+ 3533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:55.694876 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:55.694894 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:55.695472 (+ 578us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:55.696000 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:41:55.696004 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:41:55.696007 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:41:55.696268 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:41:55.696969 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:55.696977 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:55.697396 (+ 419us) client_negotiation.cc:769] Sending connection context
0810 18:41:55.697705 (+ 309us) client_negotiation.cc:240] Negotiation successful
0810 18:41:55.698177 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"thread_start_us":695,"threads_started":1}
I20260810 18:41:55.701351 11989 heartbeater.cc:347] Connected to a master server at 127.4.228.254:38533
I20260810 18:41:55.701774 11989 heartbeater.cc:464] Registering TS with master...
I20260810 18:41:55.702724 11989 heartbeater.cc:511] Master 127.4.228.254:38533 requested a full tablet report, sending...
I20260810 18:41:55.704668 11297 ts_manager.cc:194] Registered new tserver with Master: 09600514071b496db42deb2c2704ec76 (127.4.228.195:43733)
I20260810 18:41:55.705580 11297 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.195@KRBTEST.COM'} at 127.4.228.195:42399
I20260810 18:41:55.718367 5011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:41:55.739681 11994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:55.724129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50358 (local address 127.4.228.254:38533)
0810 18:41:55.724400 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:55.724406 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:55.724996 (+ 590us) server_negotiation.cc:407] Connection header received
0810 18:41:55.725148 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:55.725175 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:55.725255 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:55.725356 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:55.727181 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:55.727966 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:55.729617 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:55.730129 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:55.730271 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:55.733367 (+ 3096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:55.733378 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:55.733383 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:55.733446 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:55.736623 (+ 3177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:55.737401 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:55.737406 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:55.737408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:55.737562 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:55.738090 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:55.738094 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:55.738097 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:55.738565 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:55.738780 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:55.739202 (+ 422us) server_negotiation.cc:299] Negotiation successful
0810 18:41:55.739407 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260810 18:41:55.744959 11297 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50358:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:41:55.753024 11297 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:41:55.794627 12008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:55.767909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:42555 (local address 127.0.0.1:35230)
0810 18:41:55.770067 (+ 2158us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:55.770117 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:55.770301 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:55.783749 (+ 13448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:55.783762 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:55.783814 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:41:55.784440 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:55.784464 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:55.787096 (+ 2632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:55.787101 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:55.790554 (+ 3453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:55.790585 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:55.793308 (+ 2723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:55.793318 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:55.793559 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:55.793631 (+ 72us) client_negotiation.cc:769] Sending connection context
0810 18:41:55.793765 (+ 134us) client_negotiation.cc:240] Negotiation successful
0810 18:41:55.794010 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1908,"thread_start_us":1741,"threads_started":1}
I20260810 18:41:55.796386 12011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:55.768742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35230 (local address 127.4.228.194:42555)
0810 18:41:55.782436 (+ 13694us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:55.782452 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:55.782519 (+ 67us) server_negotiation.cc:407] Connection header received
0810 18:41:55.782686 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:55.782699 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:55.783089 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:55.784767 (+ 1678us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:41:55.785090 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:55.786921 (+ 1831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:55.790934 (+ 4013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:55.793034 (+ 2100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:55.795327 (+ 2293us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:55.795608 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:55.795695 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 18:41:55.795945 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13416,"thread_start_us":14570,"threads_started":1}
I20260810 18:41:55.802057 11689 tablet_service.cc:1467] Processing CreateTablet for tablet 3a37a400e8aa494594dfd7a5e5ea32a8 (DEFAULT_TABLE table=test-table [id=8d12d1273ba741098b5b97b72a816b63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:41:55.804340 11689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a37a400e8aa494594dfd7a5e5ea32a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:55.806089 12010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:55.768055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:43733 (local address 127.0.0.1:41708)
0810 18:41:55.770952 (+ 2897us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:55.770973 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:55.771092 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:55.796121 (+ 25029us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:55.796129 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:55.796171 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:41:55.796660 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:55.796676 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:55.799003 (+ 2327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:55.799009 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:55.802900 (+ 3891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:55.802919 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:55.805268 (+ 2349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:55.805275 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:55.805454 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:55.805493 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 18:41:55.805562 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 18:41:55.805721 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2762,"thread_start_us":2588,"threads_started":1}
I20260810 18:41:55.807202 12009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:55.768061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41708 (local address 127.4.228.195:43733)
0810 18:41:55.795278 (+ 27217us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:55.795290 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:55.795334 (+ 44us) server_negotiation.cc:407] Connection header received
0810 18:41:55.795459 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:55.795470 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:55.795831 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:55.796905 (+ 1074us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:41:55.797236 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:55.798844 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:55.803257 (+ 4413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:55.805017 (+ 1760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:55.805929 (+ 912us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:55.806213 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:55.806352 (+ 139us) server_negotiation.cc:299] Negotiation successful
0810 18:41:55.806770 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26814,"spinlock_wait_cycles":179072,"thread_start_us":26622,"threads_started":1}
I20260810 18:41:55.812155 12008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:55.771242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:35075 (local address 127.0.0.1:38154)
0810 18:41:55.794938 (+ 23696us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:55.794957 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:55.795077 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:55.798263 (+ 3186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:55.798268 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:55.798297 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:41:55.804155 (+ 5858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:55.804171 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:55.806258 (+ 2087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:55.806263 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:55.808296 (+ 2033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:55.808315 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:55.811330 (+ 3015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:55.811337 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:55.811547 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:55.811588 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 18:41:55.811679 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 18:41:55.811838 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23591,"thread_start_us":35426,"threads_started":1}
I20260810 18:41:55.811685 11892 tablet_service.cc:1467] Processing CreateTablet for tablet 3a37a400e8aa494594dfd7a5e5ea32a8 (DEFAULT_TABLE table=test-table [id=8d12d1273ba741098b5b97b72a816b63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:41:55.813091 11892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a37a400e8aa494594dfd7a5e5ea32a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:55.813275 12013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:55.771730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38154 (local address 127.4.228.193:35075)
0810 18:41:55.797382 (+ 25652us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:55.797394 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:55.797431 (+ 37us) server_negotiation.cc:407] Connection header received
0810 18:41:55.797556 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:55.797568 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:55.797856 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:55.798080 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:41:55.804356 (+ 6276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:55.806041 (+ 1685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:55.808506 (+ 2465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:55.810211 (+ 1705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:55.812399 (+ 2188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:55.812681 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:55.812761 (+ 80us) server_negotiation.cc:299] Negotiation successful
0810 18:41:55.812932 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25430,"thread_start_us":25251,"threads_started":1}
I20260810 18:41:55.817340 11480 tablet_service.cc:1467] Processing CreateTablet for tablet 3a37a400e8aa494594dfd7a5e5ea32a8 (DEFAULT_TABLE table=test-table [id=8d12d1273ba741098b5b97b72a816b63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:41:55.819015 11480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a37a400e8aa494594dfd7a5e5ea32a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:55.820008 12014 tablet_bootstrap.cc:492] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 3024406d59d74006a2d022053a7ba739: Bootstrap starting.
I20260810 18:41:55.824678 12014 tablet_bootstrap.cc:654] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 3024406d59d74006a2d022053a7ba739: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:55.825948 12014 log.cc:824] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 3024406d59d74006a2d022053a7ba739: Log is configured to *not* fsync() on all Append() calls
I20260810 18:41:55.829103 12016 tablet_bootstrap.cc:492] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 09600514071b496db42deb2c2704ec76: Bootstrap starting.
I20260810 18:41:55.829213 12014 tablet_bootstrap.cc:492] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 3024406d59d74006a2d022053a7ba739: No bootstrap required, opened a new log
I20260810 18:41:55.829510 12014 ts_tablet_manager.cc:1397] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 3024406d59d74006a2d022053a7ba739: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260810 18:41:55.833559 12016 tablet_bootstrap.cc:654] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 09600514071b496db42deb2c2704ec76: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:55.834928 12017 tablet_bootstrap.cc:492] T 3a37a400e8aa494594dfd7a5e5ea32a8 P c2a07565bdf645cbbe11a58f1a2b1bfe: Bootstrap starting.
I20260810 18:41:55.836643 12016 log.cc:824] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 09600514071b496db42deb2c2704ec76: Log is configured to *not* fsync() on all Append() calls
I20260810 18:41:55.839025 12017 tablet_bootstrap.cc:654] T 3a37a400e8aa494594dfd7a5e5ea32a8 P c2a07565bdf645cbbe11a58f1a2b1bfe: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:55.840189 12017 log.cc:824] T 3a37a400e8aa494594dfd7a5e5ea32a8 P c2a07565bdf645cbbe11a58f1a2b1bfe: Log is configured to *not* fsync() on all Append() calls
I20260810 18:41:55.840479 12016 tablet_bootstrap.cc:492] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 09600514071b496db42deb2c2704ec76: No bootstrap required, opened a new log
I20260810 18:41:55.840157 12014 raft_consensus.cc:348] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 3024406d59d74006a2d022053a7ba739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09600514071b496db42deb2c2704ec76" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 43733 } } peers { permanent_uuid: "3024406d59d74006a2d022053a7ba739" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 42555 } } peers { permanent_uuid: "c2a07565bdf645cbbe11a58f1a2b1bfe" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35075 } }
I20260810 18:41:55.840656 12014 raft_consensus.cc:374] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 3024406d59d74006a2d022053a7ba739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:55.840766 12014 raft_consensus.cc:729] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 3024406d59d74006a2d022053a7ba739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3024406d59d74006a2d022053a7ba739, State: Initialized, Role: FOLLOWER
I20260810 18:41:55.840818 12016 ts_tablet_manager.cc:1397] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 09600514071b496db42deb2c2704ec76: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.003s
I20260810 18:41:55.841537 12014 consensus_queue.cc:260] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 3024406d59d74006a2d022053a7ba739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09600514071b496db42deb2c2704ec76" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 43733 } } peers { permanent_uuid: "3024406d59d74006a2d022053a7ba739" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 42555 } } peers { permanent_uuid: "c2a07565bdf645cbbe11a58f1a2b1bfe" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35075 } }
I20260810 18:41:55.843240 11790 heartbeater.cc:503] Master 127.4.228.254:38533 was elected leader, sending a full tablet report...
I20260810 18:41:55.843663 12014 ts_tablet_manager.cc:1428] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 3024406d59d74006a2d022053a7ba739: Time spent starting tablet: real 0.014s user 0.014s sys 0.001s
I20260810 18:41:55.847076 12017 tablet_bootstrap.cc:492] T 3a37a400e8aa494594dfd7a5e5ea32a8 P c2a07565bdf645cbbe11a58f1a2b1bfe: No bootstrap required, opened a new log
I20260810 18:41:55.847436 12017 ts_tablet_manager.cc:1397] T 3a37a400e8aa494594dfd7a5e5ea32a8 P c2a07565bdf645cbbe11a58f1a2b1bfe: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.005s
W20260810 18:41:55.848941 11791 tablet.cc:2369] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 3024406d59d74006a2d022053a7ba739: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:41:55.851738 12016 raft_consensus.cc:348] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 09600514071b496db42deb2c2704ec76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09600514071b496db42deb2c2704ec76" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 43733 } } peers { permanent_uuid: "3024406d59d74006a2d022053a7ba739" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 42555 } } peers { permanent_uuid: "c2a07565bdf645cbbe11a58f1a2b1bfe" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35075 } }
I20260810 18:41:55.852159 12016 raft_consensus.cc:374] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 09600514071b496db42deb2c2704ec76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:55.852288 12016 raft_consensus.cc:729] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 09600514071b496db42deb2c2704ec76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09600514071b496db42deb2c2704ec76, State: Initialized, Role: FOLLOWER
I20260810 18:41:55.852996 12016 consensus_queue.cc:260] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 09600514071b496db42deb2c2704ec76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09600514071b496db42deb2c2704ec76" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 43733 } } peers { permanent_uuid: "3024406d59d74006a2d022053a7ba739" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 42555 } } peers { permanent_uuid: "c2a07565bdf645cbbe11a58f1a2b1bfe" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35075 } }
I20260810 18:41:55.854310 11989 heartbeater.cc:503] Master 127.4.228.254:38533 was elected leader, sending a full tablet report...
I20260810 18:41:55.854938 12016 ts_tablet_manager.cc:1428] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 09600514071b496db42deb2c2704ec76: Time spent starting tablet: real 0.014s user 0.011s sys 0.003s
I20260810 18:41:55.858074 12017 raft_consensus.cc:348] T 3a37a400e8aa494594dfd7a5e5ea32a8 P c2a07565bdf645cbbe11a58f1a2b1bfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09600514071b496db42deb2c2704ec76" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 43733 } } peers { permanent_uuid: "3024406d59d74006a2d022053a7ba739" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 42555 } } peers { permanent_uuid: "c2a07565bdf645cbbe11a58f1a2b1bfe" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35075 } }
I20260810 18:41:55.858654 12017 raft_consensus.cc:374] T 3a37a400e8aa494594dfd7a5e5ea32a8 P c2a07565bdf645cbbe11a58f1a2b1bfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:55.858800 12017 raft_consensus.cc:729] T 3a37a400e8aa494594dfd7a5e5ea32a8 P c2a07565bdf645cbbe11a58f1a2b1bfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2a07565bdf645cbbe11a58f1a2b1bfe, State: Initialized, Role: FOLLOWER
I20260810 18:41:55.859552 12017 consensus_queue.cc:260] T 3a37a400e8aa494594dfd7a5e5ea32a8 P c2a07565bdf645cbbe11a58f1a2b1bfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09600514071b496db42deb2c2704ec76" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 43733 } } peers { permanent_uuid: "3024406d59d74006a2d022053a7ba739" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 42555 } } peers { permanent_uuid: "c2a07565bdf645cbbe11a58f1a2b1bfe" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35075 } }
I20260810 18:41:55.861586 12017 ts_tablet_manager.cc:1428] T 3a37a400e8aa494594dfd7a5e5ea32a8 P c2a07565bdf645cbbe11a58f1a2b1bfe: Time spent starting tablet: real 0.014s user 0.013s sys 0.001s
W20260810 18:41:55.925432 11990 tablet.cc:2369] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 09600514071b496db42deb2c2704ec76: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:41:55.999394 11596 tablet.cc:2369] T 3a37a400e8aa494594dfd7a5e5ea32a8 P c2a07565bdf645cbbe11a58f1a2b1bfe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:41:56.124189 12025 raft_consensus.cc:482] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 09600514071b496db42deb2c2704ec76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:41:56.124583 12025 raft_consensus.cc:504] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 09600514071b496db42deb2c2704ec76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09600514071b496db42deb2c2704ec76" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 43733 } } peers { permanent_uuid: "3024406d59d74006a2d022053a7ba739" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 42555 } } peers { permanent_uuid: "c2a07565bdf645cbbe11a58f1a2b1bfe" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35075 } }
I20260810 18:41:56.126533 12025 leader_election.cc:288] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 09600514071b496db42deb2c2704ec76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3024406d59d74006a2d022053a7ba739 (127.4.228.194:42555), c2a07565bdf645cbbe11a58f1a2b1bfe (127.4.228.193:35075)
I20260810 18:41:56.134721 12011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:56.127037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:33609 (local address 127.4.228.194:42555)
0810 18:41:56.127254 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:56.127262 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:56.127302 (+ 40us) server_negotiation.cc:407] Connection header received
0810 18:41:56.127611 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:56.127617 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:56.127711 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:56.127827 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:41:56.128257 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:56.129293 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:56.131860 (+ 2567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:56.133274 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:56.133449 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:56.133529 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:56.134274 (+ 745us) server_negotiation.cc:299] Negotiation successful
0810 18:41:56.134421 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260810 18:41:56.134721 12013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:56.127091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:56519 (local address 127.4.228.193:35075)
0810 18:41:56.127375 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:56.127384 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:56.127809 (+ 425us) server_negotiation.cc:407] Connection header received
0810 18:41:56.127989 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:56.127995 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:56.128074 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:56.128147 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:41:56.128664 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:56.129634 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:56.131994 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:56.133285 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:56.133451 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:56.133529 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:56.134275 (+ 746us) server_negotiation.cc:299] Negotiation successful
0810 18:41:56.134421 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260810 18:41:56.134804 11991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:56.126890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:42555 (local address 127.4.228.195:33609)
0810 18:41:56.127197 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:56.127219 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:56.127408 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:56.127870 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:56.127877 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:56.127905 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:41:56.128077 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:56.128087 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:56.129485 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:56.129489 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:56.131686 (+ 2197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:56.131704 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:56.133621 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:56.133635 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:56.133975 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:56.134033 (+ 58us) client_negotiation.cc:769] Sending connection context
0810 18:41:56.134182 (+ 149us) client_negotiation.cc:240] Negotiation successful
0810 18:41:56.134377 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":84}
I20260810 18:41:56.135131 12029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:56.126998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:35075 (local address 127.4.228.195:56519)
0810 18:41:56.127704 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:56.127718 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:56.127845 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:56.128200 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:56.128205 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:56.128241 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:41:56.128503 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:56.128511 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:56.129797 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:56.129803 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:56.131825 (+ 2022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:41:56.131837 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:41:56.133647 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:41:56.133653 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:41:56.133968 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:56.134044 (+ 76us) client_negotiation.cc:769] Sending connection context
0810 18:41:56.134182 (+ 138us) client_negotiation.cc:240] Negotiation successful
0810 18:41:56.134377 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"mutex_wait_us":68,"thread_start_us":473,"threads_started":1}
I20260810 18:41:56.136161 11718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a37a400e8aa494594dfd7a5e5ea32a8" candidate_uuid: "09600514071b496db42deb2c2704ec76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3024406d59d74006a2d022053a7ba739" is_pre_election: true
I20260810 18:41:56.136433 11514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a37a400e8aa494594dfd7a5e5ea32a8" candidate_uuid: "09600514071b496db42deb2c2704ec76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2a07565bdf645cbbe11a58f1a2b1bfe" is_pre_election: true
I20260810 18:41:56.136898 11718 raft_consensus.cc:2454] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 3024406d59d74006a2d022053a7ba739 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09600514071b496db42deb2c2704ec76 in term 0.
I20260810 18:41:56.136983 11514 raft_consensus.cc:2454] T 3a37a400e8aa494594dfd7a5e5ea32a8 P c2a07565bdf645cbbe11a58f1a2b1bfe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09600514071b496db42deb2c2704ec76 in term 0.
I20260810 18:41:56.137687 11828 leader_election.cc:302] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 09600514071b496db42deb2c2704ec76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09600514071b496db42deb2c2704ec76, 3024406d59d74006a2d022053a7ba739; no voters:
I20260810 18:41:56.138160 12025 raft_consensus.cc:2781] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 09600514071b496db42deb2c2704ec76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:41:56.138346 12025 raft_consensus.cc:482] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 09600514071b496db42deb2c2704ec76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:41:56.138514 12025 raft_consensus.cc:3037] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 09600514071b496db42deb2c2704ec76 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:56.141165 12025 raft_consensus.cc:504] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 09600514071b496db42deb2c2704ec76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09600514071b496db42deb2c2704ec76" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 43733 } } peers { permanent_uuid: "3024406d59d74006a2d022053a7ba739" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 42555 } } peers { permanent_uuid: "c2a07565bdf645cbbe11a58f1a2b1bfe" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35075 } }
I20260810 18:41:56.141860 12025 leader_election.cc:288] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 09600514071b496db42deb2c2704ec76 [CANDIDATE]: Term 1 election: Requested vote from peers 3024406d59d74006a2d022053a7ba739 (127.4.228.194:42555), c2a07565bdf645cbbe11a58f1a2b1bfe (127.4.228.193:35075)
I20260810 18:41:56.142412 11514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a37a400e8aa494594dfd7a5e5ea32a8" candidate_uuid: "09600514071b496db42deb2c2704ec76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2a07565bdf645cbbe11a58f1a2b1bfe"
I20260810 18:41:56.142418 11718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a37a400e8aa494594dfd7a5e5ea32a8" candidate_uuid: "09600514071b496db42deb2c2704ec76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3024406d59d74006a2d022053a7ba739"
I20260810 18:41:56.142699 11514 raft_consensus.cc:3037] T 3a37a400e8aa494594dfd7a5e5ea32a8 P c2a07565bdf645cbbe11a58f1a2b1bfe [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:56.142735 11718 raft_consensus.cc:3037] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 3024406d59d74006a2d022053a7ba739 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:56.145108 11514 raft_consensus.cc:2454] T 3a37a400e8aa494594dfd7a5e5ea32a8 P c2a07565bdf645cbbe11a58f1a2b1bfe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09600514071b496db42deb2c2704ec76 in term 1.
I20260810 18:41:56.145411 11718 raft_consensus.cc:2454] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 3024406d59d74006a2d022053a7ba739 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09600514071b496db42deb2c2704ec76 in term 1.
I20260810 18:41:56.145673 11827 leader_election.cc:302] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 09600514071b496db42deb2c2704ec76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09600514071b496db42deb2c2704ec76, c2a07565bdf645cbbe11a58f1a2b1bfe; no voters:
I20260810 18:41:56.146032 12025 raft_consensus.cc:2781] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 09600514071b496db42deb2c2704ec76 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:41:56.146553 12025 raft_consensus.cc:686] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 09600514071b496db42deb2c2704ec76 [term 1 LEADER]: Becoming Leader. State: Replica: 09600514071b496db42deb2c2704ec76, State: Running, Role: LEADER
I20260810 18:41:56.147233 12025 consensus_queue.cc:237] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 09600514071b496db42deb2c2704ec76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09600514071b496db42deb2c2704ec76" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 43733 } } peers { permanent_uuid: "3024406d59d74006a2d022053a7ba739" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 42555 } } peers { permanent_uuid: "c2a07565bdf645cbbe11a58f1a2b1bfe" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35075 } }
I20260810 18:41:56.152309 11297 catalog_manager.cc:5495] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 09600514071b496db42deb2c2704ec76 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09600514071b496db42deb2c2704ec76 (127.4.228.195). New cstate: current_term: 1 leader_uuid: "09600514071b496db42deb2c2704ec76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09600514071b496db42deb2c2704ec76" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 43733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3024406d59d74006a2d022053a7ba739" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 42555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2a07565bdf645cbbe11a58f1a2b1bfe" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35075 } health_report { overall_health: UNKNOWN } } }
I20260810 18:41:56.242599 12009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:56.236633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41724 (local address 127.4.228.195:43733)
0810 18:41:56.236944 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:56.236952 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:56.237058 (+ 106us) server_negotiation.cc:407] Connection header received
0810 18:41:56.237346 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:56.237351 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:56.237438 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:56.237611 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:41:56.238040 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:56.238865 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:56.240254 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:56.240668 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:56.240816 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:56.241349 (+ 533us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:41:56.241897 (+ 548us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:41:56.242067 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:56.242149 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 18:41:56.242237 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260810 18:41:56.252295 11514 raft_consensus.cc:1261] T 3a37a400e8aa494594dfd7a5e5ea32a8 P c2a07565bdf645cbbe11a58f1a2b1bfe [term 1 FOLLOWER]: Refusing update from remote peer 09600514071b496db42deb2c2704ec76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:41:56.252307 11718 raft_consensus.cc:1261] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 3024406d59d74006a2d022053a7ba739 [term 1 FOLLOWER]: Refusing update from remote peer 09600514071b496db42deb2c2704ec76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:41:56.253633 12025 consensus_queue.cc:1035] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 09600514071b496db42deb2c2704ec76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2a07565bdf645cbbe11a58f1a2b1bfe" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:41:56.254089 12030 consensus_queue.cc:1035] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 09600514071b496db42deb2c2704ec76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3024406d59d74006a2d022053a7ba739" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 42555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:41:56.265597 12034 mvcc.cc:205] Tried to move back new op lower bound from 7317042447356747776 to 7317042446943649792. Current Snapshot: MvccSnapshot[applied={T|T < 7317042447356747776}]
I20260810 18:41:56.272253 12036 mvcc.cc:205] Tried to move back new op lower bound from 7317042447356747776 to 7317042446943649792. Current Snapshot: MvccSnapshot[applied={T|T < 7317042447356747776}]
I20260810 18:41:56.280956 11293 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50358:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:41:56.281297 11293 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50358:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:41:56.285694 11293 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3fa02cb275da4ab3b78cd49f39284d53: Sending DeleteTablet for 3 replicas of tablet 3a37a400e8aa494594dfd7a5e5ea32a8
I20260810 18:41:56.287043 11892 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a37a400e8aa494594dfd7a5e5ea32a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:41:56 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:41708
I20260810 18:41:56.287137 11480 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a37a400e8aa494594dfd7a5e5ea32a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:41:56 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:38154
I20260810 18:41:56.287175 11689 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a37a400e8aa494594dfd7a5e5ea32a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:41:56 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:35230
I20260810 18:41:56.291249 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 11391
I20260810 18:41:56.309283 12049 tablet_replica.cc:330] T 3a37a400e8aa494594dfd7a5e5ea32a8 P c2a07565bdf645cbbe11a58f1a2b1bfe: stopping tablet replica
I20260810 18:41:56.310079 12049 raft_consensus.cc:2229] T 3a37a400e8aa494594dfd7a5e5ea32a8 P c2a07565bdf645cbbe11a58f1a2b1bfe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:41:56.311439 12051 tablet_replica.cc:330] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 3024406d59d74006a2d022053a7ba739: stopping tablet replica
I20260810 18:41:56.311800 12049 raft_consensus.cc:2258] T 3a37a400e8aa494594dfd7a5e5ea32a8 P c2a07565bdf645cbbe11a58f1a2b1bfe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:41:56.311231 12050 tablet_replica.cc:330] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 09600514071b496db42deb2c2704ec76: stopping tablet replica
I20260810 18:41:56.312104 12051 raft_consensus.cc:2229] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 3024406d59d74006a2d022053a7ba739 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:41:56.312691 12050 raft_consensus.cc:2229] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 09600514071b496db42deb2c2704ec76 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:41:56.313506 12050 raft_consensus.cc:2258] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 09600514071b496db42deb2c2704ec76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:41:56.315482 12049 ts_tablet_manager.cc:1905] T 3a37a400e8aa494594dfd7a5e5ea32a8 P c2a07565bdf645cbbe11a58f1a2b1bfe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:41:56.315694 12051 raft_consensus.cc:2258] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 3024406d59d74006a2d022053a7ba739 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:41:56.315826 12050 ts_tablet_manager.cc:1905] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 09600514071b496db42deb2c2704ec76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:41:56.319829 12051 ts_tablet_manager.cc:1905] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 3024406d59d74006a2d022053a7ba739: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:41:56.324518 12049 ts_tablet_manager.cc:1918] T 3a37a400e8aa494594dfd7a5e5ea32a8 P c2a07565bdf645cbbe11a58f1a2b1bfe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:41:56.324676 12050 ts_tablet_manager.cc:1918] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 09600514071b496db42deb2c2704ec76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:41:56.324908 12050 log.cc:1196] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 09600514071b496db42deb2c2704ec76: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-2/wal/wals/3a37a400e8aa494594dfd7a5e5ea32a8
I20260810 18:41:56.325059 12049 log.cc:1196] T 3a37a400e8aa494594dfd7a5e5ea32a8 P c2a07565bdf645cbbe11a58f1a2b1bfe: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-0/wal/wals/3a37a400e8aa494594dfd7a5e5ea32a8
I20260810 18:41:56.325631 12050 ts_tablet_manager.cc:1939] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 09600514071b496db42deb2c2704ec76: Deleting consensus metadata
I20260810 18:41:56.326088 12049 ts_tablet_manager.cc:1939] T 3a37a400e8aa494594dfd7a5e5ea32a8 P c2a07565bdf645cbbe11a58f1a2b1bfe: Deleting consensus metadata
Aug 10 18:41:56 dist-test-slave-7kzw krb5kdc[11232](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387313, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.193@KRBTEST.COM
I20260810 18:41:56.328114 11278 catalog_manager.cc:4841] TS 09600514071b496db42deb2c2704ec76 (127.4.228.195:43733): tablet 3a37a400e8aa494594dfd7a5e5ea32a8 (table test-table [id=8d12d1273ba741098b5b97b72a816b63]) successfully deleted
I20260810 18:41:56.328820 11274 catalog_manager.cc:4841] TS c2a07565bdf645cbbe11a58f1a2b1bfe (127.4.228.193:35075): tablet 3a37a400e8aa494594dfd7a5e5ea32a8 (table test-table [id=8d12d1273ba741098b5b97b72a816b63]) successfully deleted
I20260810 18:41:56.333918 12051 ts_tablet_manager.cc:1918] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 3024406d59d74006a2d022053a7ba739: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:41:56.334147 12051 log.cc:1196] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 3024406d59d74006a2d022053a7ba739: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387267874959-5011-0/minicluster-data/ts-1/wal/wals/3a37a400e8aa494594dfd7a5e5ea32a8
I20260810 18:41:56.334867 12051 ts_tablet_manager.cc:1939] T 3a37a400e8aa494594dfd7a5e5ea32a8 P 3024406d59d74006a2d022053a7ba739: Deleting consensus metadata
I20260810 18:41:56.337103 11275 catalog_manager.cc:4841] TS 3024406d59d74006a2d022053a7ba739 (127.4.228.194:42555): tablet 3a37a400e8aa494594dfd7a5e5ea32a8 (table test-table [id=8d12d1273ba741098b5b97b72a816b63]) successfully deleted
I20260810 18:41:56.338349 12013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:56.293051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38166 (local address 127.4.228.193:35075)
0810 18:41:56.293327 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:56.293338 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:56.304383 (+ 11045us) server_negotiation.cc:407] Connection header received
0810 18:41:56.305166 (+ 783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:56.305178 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:56.305312 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:56.305442 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:56.311372 (+ 5930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:56.313224 (+ 1852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:56.315486 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:56.316370 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:56.316501 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:56.329205 (+ 12704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:56.329222 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:56.329245 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:56.329300 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:56.334079 (+ 4779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:56.335409 (+ 1330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:56.335422 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:56.335427 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:56.335642 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:56.336217 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:56.336224 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:56.336231 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:56.336819 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:56.337184 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:56.337621 (+ 437us) server_negotiation.cc:299] Negotiation successful
0810 18:41:56.337906 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260810 18:41:56.339202 11586 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:41:56.467979 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 11391
I20260810 18:41:56.497514 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 11601
Aug 10 18:41:56 dist-test-slave-7kzw krb5kdc[11232](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387313, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.194@KRBTEST.COM
I20260810 18:41:56.516703 12011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:56.498440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35244 (local address 127.4.228.194:42555)
0810 18:41:56.498642 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:56.498646 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:56.498666 (+ 20us) server_negotiation.cc:407] Connection header received
0810 18:41:56.498881 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:56.498885 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:56.498962 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:56.499048 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:56.500914 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:56.501632 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:56.503728 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:56.504129 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:56.504228 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:56.510031 (+ 5803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:56.510049 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:56.510068 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:56.510108 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:56.513475 (+ 3367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:56.514320 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:56.514328 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:56.514331 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:56.514540 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:56.514977 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:56.514983 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:56.514991 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:56.515423 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:56.515747 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:56.516092 (+ 345us) server_negotiation.cc:299] Negotiation successful
0810 18:41:56.516353 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 18:41:56.517488 11785 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:41:56.609310 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 11601
I20260810 18:41:56.643308 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 11801
Aug 10 18:41:56 dist-test-slave-7kzw krb5kdc[11232](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387313, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.195@KRBTEST.COM
I20260810 18:41:56.666934 12009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:56.644241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41738 (local address 127.4.228.195:43733)
0810 18:41:56.644456 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:56.644464 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:56.644622 (+ 158us) server_negotiation.cc:407] Connection header received
0810 18:41:56.644876 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:56.644881 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:56.644981 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:56.645071 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:56.647098 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:56.648078 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:56.650132 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:41:56.650865 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:41:56.650984 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:41:56.657787 (+ 6803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:56.657807 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:56.657833 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:41:56.657900 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:56.662782 (+ 4882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:56.663991 (+ 1209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:56.664004 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:56.664008 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:56.664217 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:41:56.664871 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:41:56.664877 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:41:56.664880 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:41:56.665458 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:56.665830 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:56.666208 (+ 378us) server_negotiation.cc:299] Negotiation successful
0810 18:41:56.666512 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 18:41:56.667984 11984 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:41:56.801535 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 11801
I20260810 18:41:56.833626 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 11252
I20260810 18:41:56.835163 11351 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:41:56.949978 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 11252
2026-08-10T18:41:56Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (5283 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-10T18:41:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:41:57Z Disabled control of system clock
I20260810 18:41:57.048311 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:43369
--webserver_interface=127.4.228.254
--webserver_port=0
--builtin_ntp_servers=127.4.228.212:46095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:43369
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 18:41:57.417629 12065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:41:57.417985 12065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:41:57.418074 12065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:41:57.425529 12065 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:41:57.425661 12065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:41:57.425700 12065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:41:57.425752 12065 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:41:57.425798 12065 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:41:57.436269 12065 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:46095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:43369
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:43369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:41:57.437723 12065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:41:57.439705 12065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:41:57.452734 12071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:57.452739 12075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:57.452788 12072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:57.453269 12065 server_base.cc:1034] running on GCE node
I20260810 18:41:57.454543 12065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:41:57.456054 12065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:41:57.457300 12065 hybrid_clock.cc:648] HybridClock initialized: now 1786387317457243 us; error 43 us; skew 500 ppm
I20260810 18:41:57.457803 12065 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:41:57.460439 12065 webserver.cc:460] Webserver started at http://127.4.228.254:42855/ using document root <none> and password file <none>
I20260810 18:41:57.461259 12065 fs_manager.cc:359] Metadata directory not provided
I20260810 18:41:57.461362 12065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:41:57.461758 12065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:41:57.465108 12065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "7a177a6358b64259bbf8f34a6cf7e1b2"
format_stamp: "Formatted at 2026-08-10 18:41:57 on dist-test-slave-7kzw"
server_key: "b6843fcdd2521eaf2dca24c1f8ba624f"
server_key_iv: "02f8f6fe7237f75adec545c422d7c572"
server_key_version: "encryptionkey@0"
I20260810 18:41:57.466003 12065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "7a177a6358b64259bbf8f34a6cf7e1b2"
format_stamp: "Formatted at 2026-08-10 18:41:57 on dist-test-slave-7kzw"
server_key: "b6843fcdd2521eaf2dca24c1f8ba624f"
server_key_iv: "02f8f6fe7237f75adec545c422d7c572"
server_key_version: "encryptionkey@0"
I20260810 18:41:57.471969 12065 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260810 18:41:57.476330 12085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:57.478157 12065 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260810 18:41:57.478410 12065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "7a177a6358b64259bbf8f34a6cf7e1b2"
format_stamp: "Formatted at 2026-08-10 18:41:57 on dist-test-slave-7kzw"
server_key: "b6843fcdd2521eaf2dca24c1f8ba624f"
server_key_iv: "02f8f6fe7237f75adec545c422d7c572"
server_key_version: "encryptionkey@0"
I20260810 18:41:57.478726 12065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:41:57.485810 12065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:41:57.486863 12065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:41:57.487236 12065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:41:57.513334 12065 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:43369
I20260810 18:41:57.513347 12163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:43369 every 8 connection(s)
I20260810 18:41:57.515223 12065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:41:57.517690 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 12065
I20260810 18:41:57.517982 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:41:57.519078 5011 external_mini_cluster.cc:1428] Setting key 9cae15e7f878348507e00eebd2904865
I20260810 18:41:57.521590 12165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:57.528486 12170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:57.521559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40040 (local address 127.4.228.254:43369)
0810 18:41:57.522827 (+ 1268us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:57.522839 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:57.522887 (+ 48us) server_negotiation.cc:407] Connection header received
0810 18:41:57.524510 (+ 1623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:57.524535 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:57.524985 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:57.525354 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:57.526648 (+ 1294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:57.526656 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:57.526673 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:41:57.526699 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:57.526791 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:41:57.526813 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:57.527019 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:57.527128 (+ 109us) server_negotiation.cc:299] Negotiation successful
0810 18:41:57.527431 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":740,"thread_start_us":444,"threads_started":1}
I20260810 18:41:57.535087 12165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a177a6358b64259bbf8f34a6cf7e1b2: Bootstrap starting.
I20260810 18:41:57.538762 12165 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a177a6358b64259bbf8f34a6cf7e1b2: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:57.539877 12165 log.cc:824] T 00000000000000000000000000000000 P 7a177a6358b64259bbf8f34a6cf7e1b2: Log is configured to *not* fsync() on all Append() calls
I20260810 18:41:57.543067 12165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a177a6358b64259bbf8f34a6cf7e1b2: No bootstrap required, opened a new log
I20260810 18:41:57.550736 12165 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a177a6358b64259bbf8f34a6cf7e1b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a177a6358b64259bbf8f34a6cf7e1b2" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 43369 } }
I20260810 18:41:57.551138 12165 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a177a6358b64259bbf8f34a6cf7e1b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:57.551283 12165 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a177a6358b64259bbf8f34a6cf7e1b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a177a6358b64259bbf8f34a6cf7e1b2, State: Initialized, Role: FOLLOWER
I20260810 18:41:57.552059 12165 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a177a6358b64259bbf8f34a6cf7e1b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a177a6358b64259bbf8f34a6cf7e1b2" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 43369 } }
I20260810 18:41:57.552389 12165 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a177a6358b64259bbf8f34a6cf7e1b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:41:57.552538 12165 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a177a6358b64259bbf8f34a6cf7e1b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:41:57.552841 12165 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a177a6358b64259bbf8f34a6cf7e1b2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:57.555298 12165 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a177a6358b64259bbf8f34a6cf7e1b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a177a6358b64259bbf8f34a6cf7e1b2" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 43369 } }
I20260810 18:41:57.555907 12165 leader_election.cc:302] T 00000000000000000000000000000000 P 7a177a6358b64259bbf8f34a6cf7e1b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a177a6358b64259bbf8f34a6cf7e1b2; no voters:
I20260810 18:41:57.556581 12165 leader_election.cc:288] T 00000000000000000000000000000000 P 7a177a6358b64259bbf8f34a6cf7e1b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:41:57.556933 12173 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a177a6358b64259bbf8f34a6cf7e1b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:41:57.557723 12173 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a177a6358b64259bbf8f34a6cf7e1b2 [term 1 LEADER]: Becoming Leader. State: Replica: 7a177a6358b64259bbf8f34a6cf7e1b2, State: Running, Role: LEADER
I20260810 18:41:57.558359 12173 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a177a6358b64259bbf8f34a6cf7e1b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a177a6358b64259bbf8f34a6cf7e1b2" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 43369 } }
I20260810 18:41:57.558986 12165 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a177a6358b64259bbf8f34a6cf7e1b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:41:57.561794 12176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a177a6358b64259bbf8f34a6cf7e1b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a177a6358b64259bbf8f34a6cf7e1b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a177a6358b64259bbf8f34a6cf7e1b2" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 43369 } } }
I20260810 18:41:57.561861 12177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a177a6358b64259bbf8f34a6cf7e1b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a177a6358b64259bbf8f34a6cf7e1b2. Latest consensus state: current_term: 1 leader_uuid: "7a177a6358b64259bbf8f34a6cf7e1b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a177a6358b64259bbf8f34a6cf7e1b2" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 43369 } } }
I20260810 18:41:57.562335 12176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a177a6358b64259bbf8f34a6cf7e1b2 [sys.catalog]: This master's current role is: LEADER
I20260810 18:41:57.562341 12177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a177a6358b64259bbf8f34a6cf7e1b2 [sys.catalog]: This master's current role is: LEADER
I20260810 18:41:57.563367 12192 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:41:57.569974 12192 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:41:57.579905 12192 catalog_manager.cc:1347] Generated new cluster ID: 3f7c33af44e74947bc16337ea9bfbbc1
I20260810 18:41:57.580057 12192 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:41:57.603272 12192 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:41:57.603818 12192 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:41:57.613775 12192 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a177a6358b64259bbf8f34a6cf7e1b2: Generated new TSK 0
I20260810 18:41:57.614873 12192 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:41:57.632882 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.193:0
--local_ip_for_outbound_sockets=127.4.228.193
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:43369
--builtin_ntp_servers=127.4.228.212:46095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 18:41:58.015477 12208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:41:58.015961 12208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:41:58.016124 12208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:41:58.024690 12208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:41:58.024838 12208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:41:58.025031 12208 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.193
I20260810 18:41:58.035987 12208 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:46095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:43369
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.193
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:41:58.037530 12208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:41:58.039786 12208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:41:58.054848 12213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:58.055081 12216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:58.055317 12221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:58.056480 12208 server_base.cc:1034] running on GCE node
I20260810 18:41:58.057212 12208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:41:58.058314 12208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:41:58.059602 12208 hybrid_clock.cc:648] HybridClock initialized: now 1786387318059545 us; error 89 us; skew 500 ppm
I20260810 18:41:58.060086 12208 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:41:58.063053 12208 webserver.cc:460] Webserver started at http://127.4.228.193:41815/ using document root <none> and password file <none>
I20260810 18:41:58.063856 12208 fs_manager.cc:359] Metadata directory not provided
I20260810 18:41:58.064002 12208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:41:58.064364 12208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:41:58.067669 12208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-0/data/instance:
uuid: "e78ca2699da94ca5b1d4d218b72c8a6d"
format_stamp: "Formatted at 2026-08-10 18:41:58 on dist-test-slave-7kzw"
server_key: "ab22f0bb4468d2ea6fde883ecc9623b0"
server_key_iv: "6f745fd466cb4ff2059582fe165527a7"
server_key_version: "encryptionkey@0"
I20260810 18:41:58.068547 12208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance:
uuid: "e78ca2699da94ca5b1d4d218b72c8a6d"
format_stamp: "Formatted at 2026-08-10 18:41:58 on dist-test-slave-7kzw"
server_key: "ab22f0bb4468d2ea6fde883ecc9623b0"
server_key_iv: "6f745fd466cb4ff2059582fe165527a7"
server_key_version: "encryptionkey@0"
I20260810 18:41:58.074525 12208 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 18:41:58.078708 12228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:58.080327 12208 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:41:58.080605 12208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-0/wal
uuid: "e78ca2699da94ca5b1d4d218b72c8a6d"
format_stamp: "Formatted at 2026-08-10 18:41:58 on dist-test-slave-7kzw"
server_key: "ab22f0bb4468d2ea6fde883ecc9623b0"
server_key_iv: "6f745fd466cb4ff2059582fe165527a7"
server_key_version: "encryptionkey@0"
I20260810 18:41:58.080906 12208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:41:58.088063 12208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:41:58.089032 12208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:41:58.089349 12208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:41:58.090416 12208 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:41:58.092195 12208 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:41:58.092331 12208 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:58.092492 12208 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:41:58.092599 12208 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:58.131604 12208 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.193:33575
I20260810 18:41:58.131707 12392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.193:33575 every 8 connection(s)
I20260810 18:41:58.133074 12208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
I20260810 18:41:58.143144 12398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:58.136244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:49921 (local address 127.4.228.254:43369)
0810 18:41:58.136827 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:58.136832 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:58.138300 (+ 1468us) server_negotiation.cc:407] Connection header received
0810 18:41:58.140370 (+ 2070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:58.140376 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:58.140470 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:58.140609 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:58.142145 (+ 1536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:58.142150 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:58.142153 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:41:58.142173 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:58.142223 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:41:58.142236 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:58.142393 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:58.142706 (+ 313us) server_negotiation.cc:299] Negotiation successful
0810 18:41:58.142791 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":322,"threads_started":1}
I20260810 18:41:58.143342 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 12208
I20260810 18:41:58.143579 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance
I20260810 18:41:58.144265 5011 external_mini_cluster.cc:1428] Setting key 8108da916e42f8c045f4a214e6bc099a
I20260810 18:41:58.144268 12399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:58.136672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:43369 (local address 127.4.228.193:49921)
0810 18:41:58.138068 (+ 1396us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:58.138128 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:58.140020 (+ 1892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:58.140901 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:58.140912 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:58.141024 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:58.141766 (+ 742us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:41:58.141770 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:58.141815 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:41:58.141915 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:58.142349 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:58.142364 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:58.142374 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 18:41:58.142793 (+ 419us) client_negotiation.cc:240] Negotiation successful
0810 18:41:58.143088 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":352,"threads_started":1}
I20260810 18:41:58.146342 12393 heartbeater.cc:347] Connected to a master server at 127.4.228.254:43369
I20260810 18:41:58.146724 12393 heartbeater.cc:464] Registering TS with master...
I20260810 18:41:58.147553 12393 heartbeater.cc:511] Master 127.4.228.254:43369 requested a full tablet report, sending...
I20260810 18:41:58.148034 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.194:0
--local_ip_for_outbound_sockets=127.4.228.194
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:43369
--builtin_ntp_servers=127.4.228.212:46095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260810 18:41:58.150200 12115 ts_manager.cc:194] Registered new tserver with Master: e78ca2699da94ca5b1d4d218b72c8a6d (127.4.228.193:33575)
W20260810 18:41:58.517350 12400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:41:58.517771 12400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:41:58.517913 12400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:41:58.529521 12400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:41:58.529700 12400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:41:58.529901 12400 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.194
I20260810 18:41:58.539728 12400 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:46095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:43369
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.194
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:41:58.541275 12400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:41:58.543310 12400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:41:58.557960 12408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:58.558013 12412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:58.557919 12407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:58.558977 12400 server_base.cc:1034] running on GCE node
I20260810 18:41:58.559791 12400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:41:58.560847 12400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:41:58.562071 12400 hybrid_clock.cc:648] HybridClock initialized: now 1786387318561997 us; error 67 us; skew 500 ppm
I20260810 18:41:58.562511 12400 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:41:58.565085 12400 webserver.cc:460] Webserver started at http://127.4.228.194:46207/ using document root <none> and password file <none>
I20260810 18:41:58.565889 12400 fs_manager.cc:359] Metadata directory not provided
I20260810 18:41:58.565987 12400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:41:58.566326 12400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:41:58.569492 12400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-1/data/instance:
uuid: "354ebbbe83ef4c9f818c00b38e246819"
format_stamp: "Formatted at 2026-08-10 18:41:58 on dist-test-slave-7kzw"
server_key: "aa53ebce9b9d9895fabedd1645389453"
server_key_iv: "7f9024eabf190cc3d8a697d3aabdd225"
server_key_version: "encryptionkey@0"
I20260810 18:41:58.570369 12400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance:
uuid: "354ebbbe83ef4c9f818c00b38e246819"
format_stamp: "Formatted at 2026-08-10 18:41:58 on dist-test-slave-7kzw"
server_key: "aa53ebce9b9d9895fabedd1645389453"
server_key_iv: "7f9024eabf190cc3d8a697d3aabdd225"
server_key_version: "encryptionkey@0"
I20260810 18:41:58.576326 12400 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 18:41:58.580407 12421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:58.582010 12400 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 18:41:58.582249 12400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-1/wal
uuid: "354ebbbe83ef4c9f818c00b38e246819"
format_stamp: "Formatted at 2026-08-10 18:41:58 on dist-test-slave-7kzw"
server_key: "aa53ebce9b9d9895fabedd1645389453"
server_key_iv: "7f9024eabf190cc3d8a697d3aabdd225"
server_key_version: "encryptionkey@0"
I20260810 18:41:58.582543 12400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:41:58.589181 12400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:41:58.590081 12400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:41:58.590384 12400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:41:58.591472 12400 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:41:58.593060 12400 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:41:58.593189 12400 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:58.593330 12400 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:41:58.593421 12400 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:58.633504 12400 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.194:44775
I20260810 18:41:58.633615 12583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.194:44775 every 8 connection(s)
I20260810 18:41:58.635077 12400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
I20260810 18:41:58.637425 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 12400
I20260810 18:41:58.637661 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance
I20260810 18:41:58.638181 5011 external_mini_cluster.cc:1428] Setting key 8079c1e4b1b7b2bfd094f73c6f12be79
I20260810 18:41:58.642256 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.195:0
--local_ip_for_outbound_sockets=127.4.228.195
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:43369
--builtin_ntp_servers=127.4.228.212:46095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260810 18:41:58.646202 12398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:58.638425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:58277 (local address 127.4.228.254:43369)
0810 18:41:58.638714 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:58.638721 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:58.640695 (+ 1974us) server_negotiation.cc:407] Connection header received
0810 18:41:58.643352 (+ 2657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:58.643357 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:58.643413 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:58.643506 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:58.645410 (+ 1904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:58.645415 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:58.645419 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:41:58.645440 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:58.645477 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:41:58.645491 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:58.645620 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:58.645898 (+ 278us) server_negotiation.cc:299] Negotiation successful
0810 18:41:58.645960 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149}
I20260810 18:41:58.647775 12595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:58.639130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:43369 (local address 127.4.228.194:58277)
0810 18:41:58.640295 (+ 1165us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:58.640341 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:58.642958 (+ 2617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:58.643860 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:58.643877 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:58.644036 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:58.644980 (+ 944us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:41:58.644986 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:58.645055 (+ 69us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:41:58.645195 (+ 140us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:58.645615 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:58.645628 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:58.645642 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 18:41:58.645970 (+ 328us) client_negotiation.cc:240] Negotiation successful
0810 18:41:58.646292 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":386,"threads_started":1}
I20260810 18:41:58.649972 12584 heartbeater.cc:347] Connected to a master server at 127.4.228.254:43369
I20260810 18:41:58.650388 12584 heartbeater.cc:464] Registering TS with master...
I20260810 18:41:58.651551 12584 heartbeater.cc:511] Master 127.4.228.254:43369 requested a full tablet report, sending...
I20260810 18:41:58.653360 12115 ts_manager.cc:194] Registered new tserver with Master: 354ebbbe83ef4c9f818c00b38e246819 (127.4.228.194:44775)
W20260810 18:41:59.013027 12596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:41:59.013477 12596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:41:59.013664 12596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:41:59.021281 12596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:41:59.021478 12596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:41:59.021672 12596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.195
I20260810 18:41:59.031348 12596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:46095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:43369
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.195
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:41:59.032881 12596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:41:59.034891 12596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:41:59.049705 12602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:59.049849 12603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:41:59.049823 12605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:41:59.051113 12596 server_base.cc:1034] running on GCE node
I20260810 18:41:59.051847 12596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:41:59.052887 12596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:41:59.054122 12596 hybrid_clock.cc:648] HybridClock initialized: now 1786387319054064 us; error 49 us; skew 500 ppm
I20260810 18:41:59.054450 12596 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:41:59.057371 12596 webserver.cc:460] Webserver started at http://127.4.228.195:40083/ using document root <none> and password file <none>
I20260810 18:41:59.058190 12596 fs_manager.cc:359] Metadata directory not provided
I20260810 18:41:59.058295 12596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:41:59.058697 12596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:41:59.062196 12596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-2/data/instance:
uuid: "15a1d63e41d74551812c0945bca71401"
format_stamp: "Formatted at 2026-08-10 18:41:59 on dist-test-slave-7kzw"
server_key: "6e9d47d3e4989682a3de896c3febdef2"
server_key_iv: "c33e716107a71e97b02702dd0c5068c1"
server_key_version: "encryptionkey@0"
I20260810 18:41:59.063280 12596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance:
uuid: "15a1d63e41d74551812c0945bca71401"
format_stamp: "Formatted at 2026-08-10 18:41:59 on dist-test-slave-7kzw"
server_key: "6e9d47d3e4989682a3de896c3febdef2"
server_key_iv: "c33e716107a71e97b02702dd0c5068c1"
server_key_version: "encryptionkey@0"
I20260810 18:41:59.069569 12596 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 18:41:59.074163 12615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:59.076035 12596 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 18:41:59.076284 12596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-2/wal
uuid: "15a1d63e41d74551812c0945bca71401"
format_stamp: "Formatted at 2026-08-10 18:41:59 on dist-test-slave-7kzw"
server_key: "6e9d47d3e4989682a3de896c3febdef2"
server_key_iv: "c33e716107a71e97b02702dd0c5068c1"
server_key_version: "encryptionkey@0"
I20260810 18:41:59.076591 12596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:41:59.083637 12596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:41:59.084659 12596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:41:59.084992 12596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:41:59.086162 12596 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:41:59.088084 12596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:41:59.088232 12596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:59.088335 12596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:41:59.088398 12596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:41:59.128180 12596 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.195:46315
I20260810 18:41:59.128316 12775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.195:46315 every 8 connection(s)
I20260810 18:41:59.129707 12596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
I20260810 18:41:59.132767 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 12596
I20260810 18:41:59.133004 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance
I20260810 18:41:59.133677 5011 external_mini_cluster.cc:1428] Setting key 44b76df9ceb2bca889f4a34615c1f4d8
I20260810 18:41:59.140797 12398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:59.133276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:39755 (local address 127.4.228.254:43369)
0810 18:41:59.133599 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:59.133606 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:59.135536 (+ 1930us) server_negotiation.cc:407] Connection header received
0810 18:41:59.138118 (+ 2582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:59.138124 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:59.138208 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:59.138308 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:59.139964 (+ 1656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:59.139969 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:59.139973 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:41:59.139993 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:59.140030 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:41:59.140044 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:59.140163 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:59.140465 (+ 302us) server_negotiation.cc:299] Negotiation successful
0810 18:41:59.140524 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169}
I20260810 18:41:59.141913 12779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:59.133991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:43369 (local address 127.4.228.195:39755)
0810 18:41:59.135346 (+ 1355us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:59.135399 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:59.137814 (+ 2415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:59.138688 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:59.138698 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:59.138807 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:59.139574 (+ 767us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:41:59.139577 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:59.139634 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:41:59.139751 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:59.140198 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:59.140208 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:59.140217 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 18:41:59.140494 (+ 277us) client_negotiation.cc:240] Negotiation successful
0810 18:41:59.140771 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":724,"thread_start_us":473,"threads_started":1}
I20260810 18:41:59.143640 12777 heartbeater.cc:347] Connected to a master server at 127.4.228.254:43369
I20260810 18:41:59.144110 12777 heartbeater.cc:464] Registering TS with master...
I20260810 18:41:59.144856 12777 heartbeater.cc:511] Master 127.4.228.254:43369 requested a full tablet report, sending...
I20260810 18:41:59.146589 12115 ts_manager.cc:194] Registered new tserver with Master: 15a1d63e41d74551812c0945bca71401 (127.4.228.195:46315)
I20260810 18:41:59.150979 5011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:41:59.153934 12393 heartbeater.cc:503] Master 127.4.228.254:43369 was elected leader, sending a full tablet report...
I20260810 18:41:59.159874 12398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:59.156593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40046 (local address 127.4.228.254:43369)
0810 18:41:59.156828 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:59.156836 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:59.157447 (+ 611us) server_negotiation.cc:407] Connection header received
0810 18:41:59.157631 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:59.157635 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:59.157695 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:59.157765 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:59.158920 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:59.158925 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:59.158928 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:41:59.158947 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:59.158986 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:41:59.159037 (+ 51us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:59.159177 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:59.159394 (+ 217us) server_negotiation.cc:299] Negotiation successful
0810 18:41:59.159505 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 18:41:59.164680 12115 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40046:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:41:59.171904 12115 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:41:59.202160 12799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:59.188597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60008 (local address 127.4.228.195:46315)
0810 18:41:59.198210 (+ 9613us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:59.198235 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:59.198273 (+ 38us) server_negotiation.cc:407] Connection header received
0810 18:41:59.198354 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:59.198362 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:59.198619 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:59.198816 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:59.200992 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:59.200999 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:59.201010 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:41:59.201024 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:59.201067 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:41:59.201081 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:59.201309 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:59.201592 (+ 283us) server_negotiation.cc:299] Negotiation successful
0810 18:41:59.201732 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9339,"thread_start_us":9143,"threads_started":1}
I20260810 18:41:59.202230 12798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:59.188313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:46315 (local address 127.0.0.1:60008)
0810 18:41:59.189314 (+ 1001us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:59.189365 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:59.189535 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:59.198884 (+ 9349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:59.198892 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:59.199003 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:59.200037 (+ 1034us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:41:59.200044 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:59.200082 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:41:59.200192 (+ 110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:59.201220 (+ 1028us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:59.201234 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:59.201245 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 18:41:59.201657 (+ 412us) client_negotiation.cc:240] Negotiation successful
0810 18:41:59.201745 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":387,"threads_started":1}
I20260810 18:41:59.202804 12800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:59.189635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42224 (local address 127.4.228.194:44775)
0810 18:41:59.191980 (+ 2345us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:59.192007 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:59.196855 (+ 4848us) server_negotiation.cc:407] Connection header received
0810 18:41:59.197583 (+ 728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:59.197599 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:59.197967 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:59.198125 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:59.200907 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:59.200918 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:59.200935 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:41:59.200974 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:59.201052 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:41:59.201081 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:59.201245 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:59.201816 (+ 571us) server_negotiation.cc:299] Negotiation successful
0810 18:41:59.201935 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2091,"thread_start_us":301,"threads_started":1}
I20260810 18:41:59.202908 12801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:59.189780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:33575 (local address 127.0.0.1:55698)
0810 18:41:59.197359 (+ 7579us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:59.197380 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:59.197477 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:59.199155 (+ 1678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:59.199162 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:59.199188 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:59.200386 (+ 1198us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:41:59.200388 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:59.200402 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:41:59.200437 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:59.202065 (+ 1628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:59.202071 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:59.202083 (+ 12us) client_negotiation.cc:769] Sending connection context
0810 18:41:59.202249 (+ 166us) client_negotiation.cc:240] Negotiation successful
0810 18:41:59.202320 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7440,"thread_start_us":6524,"threads_started":1}
I20260810 18:41:59.203038 12803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:59.189084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:44775 (local address 127.0.0.1:42224)
0810 18:41:59.196667 (+ 7583us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:59.196697 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:59.197376 (+ 679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:59.199472 (+ 2096us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:59.199480 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:59.199516 (+ 36us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:59.200645 (+ 1129us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:41:59.200649 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:59.200668 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:41:59.200718 (+ 50us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:59.201311 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:59.201317 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:59.201321 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 18:41:59.201595 (+ 274us) client_negotiation.cc:240] Negotiation successful
0810 18:41:59.202418 (+ 823us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7395,"mutex_wait_us":676,"thread_start_us":7988,"threads_started":1}
I20260810 18:41:59.204041 12804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:59.190329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55698 (local address 127.4.228.193:33575)
0810 18:41:59.198599 (+ 8270us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:59.198606 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:59.198632 (+ 26us) server_negotiation.cc:407] Connection header received
0810 18:41:59.198694 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:59.198699 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:59.198933 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:59.200638 (+ 1705us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:59.201785 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:59.201797 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:59.201811 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:41:59.201833 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:59.201895 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:41:59.201918 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:59.203426 (+ 1508us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:59.203516 (+ 90us) server_negotiation.cc:299] Negotiation successful
0810 18:41:59.203634 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8089,"thread_start_us":7911,"threads_started":1}
I20260810 18:41:59.207167 12484 tablet_service.cc:1467] Processing CreateTablet for tablet 4ba89426ec8949f4886a7612c7c3973e (DEFAULT_TABLE table=test-table [id=c56b8f3d451444218f616e4c18ef38b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:41:59.207087 12686 tablet_service.cc:1467] Processing CreateTablet for tablet 4ba89426ec8949f4886a7612c7c3973e (DEFAULT_TABLE table=test-table [id=c56b8f3d451444218f616e4c18ef38b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:41:59.207769 12297 tablet_service.cc:1467] Processing CreateTablet for tablet 4ba89426ec8949f4886a7612c7c3973e (DEFAULT_TABLE table=test-table [id=c56b8f3d451444218f616e4c18ef38b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:41:59.208756 12484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ba89426ec8949f4886a7612c7c3973e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:59.208801 12686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ba89426ec8949f4886a7612c7c3973e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:59.209136 12297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ba89426ec8949f4886a7612c7c3973e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:41:59.222827 12805 tablet_bootstrap.cc:492] T 4ba89426ec8949f4886a7612c7c3973e P 354ebbbe83ef4c9f818c00b38e246819: Bootstrap starting.
I20260810 18:41:59.226810 12805 tablet_bootstrap.cc:654] T 4ba89426ec8949f4886a7612c7c3973e P 354ebbbe83ef4c9f818c00b38e246819: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:59.227427 12806 tablet_bootstrap.cc:492] T 4ba89426ec8949f4886a7612c7c3973e P e78ca2699da94ca5b1d4d218b72c8a6d: Bootstrap starting.
I20260810 18:41:59.229982 12805 log.cc:824] T 4ba89426ec8949f4886a7612c7c3973e P 354ebbbe83ef4c9f818c00b38e246819: Log is configured to *not* fsync() on all Append() calls
I20260810 18:41:59.231308 12807 tablet_bootstrap.cc:492] T 4ba89426ec8949f4886a7612c7c3973e P 15a1d63e41d74551812c0945bca71401: Bootstrap starting.
I20260810 18:41:59.232087 12806 tablet_bootstrap.cc:654] T 4ba89426ec8949f4886a7612c7c3973e P e78ca2699da94ca5b1d4d218b72c8a6d: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:59.233558 12806 log.cc:824] T 4ba89426ec8949f4886a7612c7c3973e P e78ca2699da94ca5b1d4d218b72c8a6d: Log is configured to *not* fsync() on all Append() calls
I20260810 18:41:59.234863 12805 tablet_bootstrap.cc:492] T 4ba89426ec8949f4886a7612c7c3973e P 354ebbbe83ef4c9f818c00b38e246819: No bootstrap required, opened a new log
I20260810 18:41:59.235209 12807 tablet_bootstrap.cc:654] T 4ba89426ec8949f4886a7612c7c3973e P 15a1d63e41d74551812c0945bca71401: Neither blocks nor log segments found. Creating new log.
I20260810 18:41:59.235220 12805 ts_tablet_manager.cc:1397] T 4ba89426ec8949f4886a7612c7c3973e P 354ebbbe83ef4c9f818c00b38e246819: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.006s
I20260810 18:41:59.236565 12807 log.cc:824] T 4ba89426ec8949f4886a7612c7c3973e P 15a1d63e41d74551812c0945bca71401: Log is configured to *not* fsync() on all Append() calls
I20260810 18:41:59.237134 12806 tablet_bootstrap.cc:492] T 4ba89426ec8949f4886a7612c7c3973e P e78ca2699da94ca5b1d4d218b72c8a6d: No bootstrap required, opened a new log
I20260810 18:41:59.237463 12806 ts_tablet_manager.cc:1397] T 4ba89426ec8949f4886a7612c7c3973e P e78ca2699da94ca5b1d4d218b72c8a6d: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260810 18:41:59.239674 12807 tablet_bootstrap.cc:492] T 4ba89426ec8949f4886a7612c7c3973e P 15a1d63e41d74551812c0945bca71401: No bootstrap required, opened a new log
I20260810 18:41:59.240000 12807 ts_tablet_manager.cc:1397] T 4ba89426ec8949f4886a7612c7c3973e P 15a1d63e41d74551812c0945bca71401: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260810 18:41:59.248363 12807 raft_consensus.cc:348] T 4ba89426ec8949f4886a7612c7c3973e P 15a1d63e41d74551812c0945bca71401 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a1d63e41d74551812c0945bca71401" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 46315 } } peers { permanent_uuid: "e78ca2699da94ca5b1d4d218b72c8a6d" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 33575 } } peers { permanent_uuid: "354ebbbe83ef4c9f818c00b38e246819" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44775 } }
I20260810 18:41:59.248816 12807 raft_consensus.cc:374] T 4ba89426ec8949f4886a7612c7c3973e P 15a1d63e41d74551812c0945bca71401 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:59.248910 12807 raft_consensus.cc:729] T 4ba89426ec8949f4886a7612c7c3973e P 15a1d63e41d74551812c0945bca71401 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15a1d63e41d74551812c0945bca71401, State: Initialized, Role: FOLLOWER
I20260810 18:41:59.248572 12805 raft_consensus.cc:348] T 4ba89426ec8949f4886a7612c7c3973e P 354ebbbe83ef4c9f818c00b38e246819 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a1d63e41d74551812c0945bca71401" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 46315 } } peers { permanent_uuid: "e78ca2699da94ca5b1d4d218b72c8a6d" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 33575 } } peers { permanent_uuid: "354ebbbe83ef4c9f818c00b38e246819" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44775 } }
I20260810 18:41:59.249085 12805 raft_consensus.cc:374] T 4ba89426ec8949f4886a7612c7c3973e P 354ebbbe83ef4c9f818c00b38e246819 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:59.249204 12805 raft_consensus.cc:729] T 4ba89426ec8949f4886a7612c7c3973e P 354ebbbe83ef4c9f818c00b38e246819 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 354ebbbe83ef4c9f818c00b38e246819, State: Initialized, Role: FOLLOWER
I20260810 18:41:59.249483 12806 raft_consensus.cc:348] T 4ba89426ec8949f4886a7612c7c3973e P e78ca2699da94ca5b1d4d218b72c8a6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a1d63e41d74551812c0945bca71401" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 46315 } } peers { permanent_uuid: "e78ca2699da94ca5b1d4d218b72c8a6d" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 33575 } } peers { permanent_uuid: "354ebbbe83ef4c9f818c00b38e246819" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44775 } }
I20260810 18:41:59.249720 12807 consensus_queue.cc:260] T 4ba89426ec8949f4886a7612c7c3973e P 15a1d63e41d74551812c0945bca71401 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a1d63e41d74551812c0945bca71401" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 46315 } } peers { permanent_uuid: "e78ca2699da94ca5b1d4d218b72c8a6d" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 33575 } } peers { permanent_uuid: "354ebbbe83ef4c9f818c00b38e246819" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44775 } }
I20260810 18:41:59.249958 12806 raft_consensus.cc:374] T 4ba89426ec8949f4886a7612c7c3973e P e78ca2699da94ca5b1d4d218b72c8a6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:41:59.249904 12805 consensus_queue.cc:260] T 4ba89426ec8949f4886a7612c7c3973e P 354ebbbe83ef4c9f818c00b38e246819 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a1d63e41d74551812c0945bca71401" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 46315 } } peers { permanent_uuid: "e78ca2699da94ca5b1d4d218b72c8a6d" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 33575 } } peers { permanent_uuid: "354ebbbe83ef4c9f818c00b38e246819" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44775 } }
I20260810 18:41:59.250181 12806 raft_consensus.cc:729] T 4ba89426ec8949f4886a7612c7c3973e P e78ca2699da94ca5b1d4d218b72c8a6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e78ca2699da94ca5b1d4d218b72c8a6d, State: Initialized, Role: FOLLOWER
I20260810 18:41:59.251260 12806 consensus_queue.cc:260] T 4ba89426ec8949f4886a7612c7c3973e P e78ca2699da94ca5b1d4d218b72c8a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a1d63e41d74551812c0945bca71401" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 46315 } } peers { permanent_uuid: "e78ca2699da94ca5b1d4d218b72c8a6d" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 33575 } } peers { permanent_uuid: "354ebbbe83ef4c9f818c00b38e246819" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44775 } }
I20260810 18:41:59.252178 12584 heartbeater.cc:503] Master 127.4.228.254:43369 was elected leader, sending a full tablet report...
I20260810 18:41:59.259377 12806 ts_tablet_manager.cc:1428] T 4ba89426ec8949f4886a7612c7c3973e P e78ca2699da94ca5b1d4d218b72c8a6d: Time spent starting tablet: real 0.022s user 0.017s sys 0.006s
I20260810 18:41:59.259819 12805 ts_tablet_manager.cc:1428] T 4ba89426ec8949f4886a7612c7c3973e P 354ebbbe83ef4c9f818c00b38e246819: Time spent starting tablet: real 0.024s user 0.013s sys 0.003s
I20260810 18:41:59.260996 12777 heartbeater.cc:503] Master 127.4.228.254:43369 was elected leader, sending a full tablet report...
I20260810 18:41:59.261579 12807 ts_tablet_manager.cc:1428] T 4ba89426ec8949f4886a7612c7c3973e P 15a1d63e41d74551812c0945bca71401: Time spent starting tablet: real 0.021s user 0.020s sys 0.000s
I20260810 18:41:59.295740 12818 raft_consensus.cc:482] T 4ba89426ec8949f4886a7612c7c3973e P e78ca2699da94ca5b1d4d218b72c8a6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:41:59.296178 12818 raft_consensus.cc:504] T 4ba89426ec8949f4886a7612c7c3973e P e78ca2699da94ca5b1d4d218b72c8a6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a1d63e41d74551812c0945bca71401" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 46315 } } peers { permanent_uuid: "e78ca2699da94ca5b1d4d218b72c8a6d" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 33575 } } peers { permanent_uuid: "354ebbbe83ef4c9f818c00b38e246819" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44775 } }
I20260810 18:41:59.298988 12818 leader_election.cc:288] T 4ba89426ec8949f4886a7612c7c3973e P e78ca2699da94ca5b1d4d218b72c8a6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15a1d63e41d74551812c0945bca71401 (127.4.228.195:46315), 354ebbbe83ef4c9f818c00b38e246819 (127.4.228.194:44775)
I20260810 18:41:59.302868 12800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:59.299769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:34603 (local address 127.4.228.194:44775)
0810 18:41:59.300008 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:59.300018 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:59.300376 (+ 358us) server_negotiation.cc:407] Connection header received
0810 18:41:59.300590 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:59.300595 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:59.300660 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:59.300799 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:59.301963 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:59.301987 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:59.301991 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:41:59.302010 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:59.302044 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:41:59.302058 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:59.302235 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:59.302394 (+ 159us) server_negotiation.cc:299] Negotiation successful
0810 18:41:59.302513 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 18:41:59.302911 12799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:59.299344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:59505 (local address 127.4.228.195:46315)
0810 18:41:59.299584 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:59.299595 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:59.300700 (+ 1105us) server_negotiation.cc:407] Connection header received
0810 18:41:59.300897 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:59.300902 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:59.300957 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:59.301076 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:59.302168 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:59.302174 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:59.302179 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:41:59.302214 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:59.302246 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:41:59.302258 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:59.302406 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:59.302523 (+ 117us) server_negotiation.cc:299] Negotiation successful
0810 18:41:59.302597 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 18:41:59.303062 12819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:59.299325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:44775 (local address 127.4.228.193:34603)
0810 18:41:59.300199 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:59.300238 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:59.300426 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:59.300841 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:59.300848 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:59.300875 (+ 27us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:59.301716 (+ 841us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:41:59.301719 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:59.301735 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:41:59.301777 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:59.302235 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:59.302241 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:59.302249 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 18:41:59.302440 (+ 191us) client_negotiation.cc:240] Negotiation successful
0810 18:41:59.302524 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":694,"thread_start_us":491,"threads_started":1}
I20260810 18:41:59.303223 12820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:59.299459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:46315 (local address 127.4.228.193:59505)
0810 18:41:59.300581 (+ 1122us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:59.300602 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:41:59.300735 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:41:59.301119 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:41:59.301126 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:41:59.301155 (+ 29us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:41:59.301928 (+ 773us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:41:59.301931 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:41:59.301945 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:41:59.301986 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:41:59.302369 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:41:59.302374 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:41:59.302383 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 18:41:59.302582 (+ 199us) client_negotiation.cc:240] Negotiation successful
0810 18:41:59.302636 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":977,"thread_start_us":726,"threads_started":1}
I20260810 18:41:59.304267 12509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ba89426ec8949f4886a7612c7c3973e" candidate_uuid: "e78ca2699da94ca5b1d4d218b72c8a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "354ebbbe83ef4c9f818c00b38e246819" is_pre_election: true
I20260810 18:41:59.304266 12714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ba89426ec8949f4886a7612c7c3973e" candidate_uuid: "e78ca2699da94ca5b1d4d218b72c8a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15a1d63e41d74551812c0945bca71401" is_pre_election: true
I20260810 18:41:59.304837 12714 raft_consensus.cc:2454] T 4ba89426ec8949f4886a7612c7c3973e P 15a1d63e41d74551812c0945bca71401 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e78ca2699da94ca5b1d4d218b72c8a6d in term 0.
I20260810 18:41:59.304854 12509 raft_consensus.cc:2454] T 4ba89426ec8949f4886a7612c7c3973e P 354ebbbe83ef4c9f818c00b38e246819 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e78ca2699da94ca5b1d4d218b72c8a6d in term 0.
I20260810 18:41:59.305598 12230 leader_election.cc:302] T 4ba89426ec8949f4886a7612c7c3973e P e78ca2699da94ca5b1d4d218b72c8a6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15a1d63e41d74551812c0945bca71401, e78ca2699da94ca5b1d4d218b72c8a6d; no voters:
I20260810 18:41:59.306059 12818 raft_consensus.cc:2781] T 4ba89426ec8949f4886a7612c7c3973e P e78ca2699da94ca5b1d4d218b72c8a6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:41:59.306242 12818 raft_consensus.cc:482] T 4ba89426ec8949f4886a7612c7c3973e P e78ca2699da94ca5b1d4d218b72c8a6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:41:59.306360 12818 raft_consensus.cc:3037] T 4ba89426ec8949f4886a7612c7c3973e P e78ca2699da94ca5b1d4d218b72c8a6d [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:59.308858 12818 raft_consensus.cc:504] T 4ba89426ec8949f4886a7612c7c3973e P e78ca2699da94ca5b1d4d218b72c8a6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a1d63e41d74551812c0945bca71401" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 46315 } } peers { permanent_uuid: "e78ca2699da94ca5b1d4d218b72c8a6d" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 33575 } } peers { permanent_uuid: "354ebbbe83ef4c9f818c00b38e246819" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44775 } }
I20260810 18:41:59.309676 12818 leader_election.cc:288] T 4ba89426ec8949f4886a7612c7c3973e P e78ca2699da94ca5b1d4d218b72c8a6d [CANDIDATE]: Term 1 election: Requested vote from peers 15a1d63e41d74551812c0945bca71401 (127.4.228.195:46315), 354ebbbe83ef4c9f818c00b38e246819 (127.4.228.194:44775)
I20260810 18:41:59.310295 12509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ba89426ec8949f4886a7612c7c3973e" candidate_uuid: "e78ca2699da94ca5b1d4d218b72c8a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "354ebbbe83ef4c9f818c00b38e246819"
I20260810 18:41:59.310304 12714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ba89426ec8949f4886a7612c7c3973e" candidate_uuid: "e78ca2699da94ca5b1d4d218b72c8a6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15a1d63e41d74551812c0945bca71401"
I20260810 18:41:59.310556 12509 raft_consensus.cc:3037] T 4ba89426ec8949f4886a7612c7c3973e P 354ebbbe83ef4c9f818c00b38e246819 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:59.310611 12714 raft_consensus.cc:3037] T 4ba89426ec8949f4886a7612c7c3973e P 15a1d63e41d74551812c0945bca71401 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:41:59.313105 12509 raft_consensus.cc:2454] T 4ba89426ec8949f4886a7612c7c3973e P 354ebbbe83ef4c9f818c00b38e246819 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e78ca2699da94ca5b1d4d218b72c8a6d in term 1.
I20260810 18:41:59.313217 12714 raft_consensus.cc:2454] T 4ba89426ec8949f4886a7612c7c3973e P 15a1d63e41d74551812c0945bca71401 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e78ca2699da94ca5b1d4d218b72c8a6d in term 1.
I20260810 18:41:59.313639 12231 leader_election.cc:302] T 4ba89426ec8949f4886a7612c7c3973e P e78ca2699da94ca5b1d4d218b72c8a6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 354ebbbe83ef4c9f818c00b38e246819, e78ca2699da94ca5b1d4d218b72c8a6d; no voters:
I20260810 18:41:59.314025 12818 raft_consensus.cc:2781] T 4ba89426ec8949f4886a7612c7c3973e P e78ca2699da94ca5b1d4d218b72c8a6d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:41:59.314684 12818 raft_consensus.cc:686] T 4ba89426ec8949f4886a7612c7c3973e P e78ca2699da94ca5b1d4d218b72c8a6d [term 1 LEADER]: Becoming Leader. State: Replica: e78ca2699da94ca5b1d4d218b72c8a6d, State: Running, Role: LEADER
I20260810 18:41:59.315317 12818 consensus_queue.cc:237] T 4ba89426ec8949f4886a7612c7c3973e P e78ca2699da94ca5b1d4d218b72c8a6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a1d63e41d74551812c0945bca71401" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 46315 } } peers { permanent_uuid: "e78ca2699da94ca5b1d4d218b72c8a6d" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 33575 } } peers { permanent_uuid: "354ebbbe83ef4c9f818c00b38e246819" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44775 } }
I20260810 18:41:59.320749 12116 catalog_manager.cc:5495] T 4ba89426ec8949f4886a7612c7c3973e P e78ca2699da94ca5b1d4d218b72c8a6d reported cstate change: term changed from 0 to 1, leader changed from <none> to e78ca2699da94ca5b1d4d218b72c8a6d (127.4.228.193). New cstate: current_term: 1 leader_uuid: "e78ca2699da94ca5b1d4d218b72c8a6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15a1d63e41d74551812c0945bca71401" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 46315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e78ca2699da94ca5b1d4d218b72c8a6d" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 33575 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "354ebbbe83ef4c9f818c00b38e246819" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44775 } health_report { overall_health: UNKNOWN } } }
I20260810 18:41:59.356945 12804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:59.354675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55704 (local address 127.4.228.193:33575)
0810 18:41:59.354869 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:59.354874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:59.354894 (+ 20us) server_negotiation.cc:407] Connection header received
0810 18:41:59.354997 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:59.355000 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:59.355087 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:59.355182 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:59.356232 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:59.356237 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:59.356241 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:41:59.356260 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:59.356300 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:41:59.356314 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:59.356441 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:59.356645 (+ 204us) server_negotiation.cc:299] Negotiation successful
0810 18:41:59.356706 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 18:41:59.366958 12714 raft_consensus.cc:1261] T 4ba89426ec8949f4886a7612c7c3973e P 15a1d63e41d74551812c0945bca71401 [term 1 FOLLOWER]: Refusing update from remote peer e78ca2699da94ca5b1d4d218b72c8a6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:41:59.366958 12509 raft_consensus.cc:1261] T 4ba89426ec8949f4886a7612c7c3973e P 354ebbbe83ef4c9f818c00b38e246819 [term 1 FOLLOWER]: Refusing update from remote peer e78ca2699da94ca5b1d4d218b72c8a6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:41:59.368288 12822 consensus_queue.cc:1035] T 4ba89426ec8949f4886a7612c7c3973e P e78ca2699da94ca5b1d4d218b72c8a6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "15a1d63e41d74551812c0945bca71401" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 46315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:41:59.368729 12818 consensus_queue.cc:1035] T 4ba89426ec8949f4886a7612c7c3973e P e78ca2699da94ca5b1d4d218b72c8a6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "354ebbbe83ef4c9f818c00b38e246819" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 18:41:59.381768 12778 tablet.cc:2369] T 4ba89426ec8949f4886a7612c7c3973e P 15a1d63e41d74551812c0945bca71401: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:41:59.382943 12827 mvcc.cc:205] Tried to move back new op lower bound from 7317042460114935808 to 7317042459920351232. Current Snapshot: MvccSnapshot[applied={T|T < 7317042460114935808}]
I20260810 18:41:59.398947 12115 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40046:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:41:59.399276 12115 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40046:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:41:59.403438 12115 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7a177a6358b64259bbf8f34a6cf7e1b2: Sending DeleteTablet for 3 replicas of tablet 4ba89426ec8949f4886a7612c7c3973e
I20260810 18:41:59.404814 12484 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ba89426ec8949f4886a7612c7c3973e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:41:59 UTC) from {username='slave'} at 127.0.0.1:42224
I20260810 18:41:59.404801 12686 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ba89426ec8949f4886a7612c7c3973e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:41:59 UTC) from {username='slave'} at 127.0.0.1:60008
I20260810 18:41:59.404851 12297 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ba89426ec8949f4886a7612c7c3973e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:41:59 UTC) from {username='slave'} at 127.0.0.1:55698
I20260810 18:41:59.408740 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 12208
I20260810 18:41:59.417351 12843 tablet_replica.cc:330] T 4ba89426ec8949f4886a7612c7c3973e P 354ebbbe83ef4c9f818c00b38e246819: stopping tablet replica
I20260810 18:41:59.417593 12842 tablet_replica.cc:330] T 4ba89426ec8949f4886a7612c7c3973e P 15a1d63e41d74551812c0945bca71401: stopping tablet replica
I20260810 18:41:59.418082 12842 raft_consensus.cc:2229] T 4ba89426ec8949f4886a7612c7c3973e P 15a1d63e41d74551812c0945bca71401 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:41:59.418082 12843 raft_consensus.cc:2229] T 4ba89426ec8949f4886a7612c7c3973e P 354ebbbe83ef4c9f818c00b38e246819 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:41:59.418715 12842 raft_consensus.cc:2258] T 4ba89426ec8949f4886a7612c7c3973e P 15a1d63e41d74551812c0945bca71401 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:41:59.418708 12843 raft_consensus.cc:2258] T 4ba89426ec8949f4886a7612c7c3973e P 354ebbbe83ef4c9f818c00b38e246819 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:41:59.421584 12843 ts_tablet_manager.cc:1905] T 4ba89426ec8949f4886a7612c7c3973e P 354ebbbe83ef4c9f818c00b38e246819: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:41:59.422729 12841 tablet_replica.cc:330] T 4ba89426ec8949f4886a7612c7c3973e P e78ca2699da94ca5b1d4d218b72c8a6d: stopping tablet replica
I20260810 18:41:59.423389 12841 raft_consensus.cc:2229] T 4ba89426ec8949f4886a7612c7c3973e P e78ca2699da94ca5b1d4d218b72c8a6d [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:41:59.424206 12841 raft_consensus.cc:2258] T 4ba89426ec8949f4886a7612c7c3973e P e78ca2699da94ca5b1d4d218b72c8a6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:41:59.426846 12841 ts_tablet_manager.cc:1905] T 4ba89426ec8949f4886a7612c7c3973e P e78ca2699da94ca5b1d4d218b72c8a6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:41:59.426957 12842 ts_tablet_manager.cc:1905] T 4ba89426ec8949f4886a7612c7c3973e P 15a1d63e41d74551812c0945bca71401: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:41:59.429522 12843 ts_tablet_manager.cc:1918] T 4ba89426ec8949f4886a7612c7c3973e P 354ebbbe83ef4c9f818c00b38e246819: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:41:59.429730 12843 log.cc:1196] T 4ba89426ec8949f4886a7612c7c3973e P 354ebbbe83ef4c9f818c00b38e246819: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-1/wal/wals/4ba89426ec8949f4886a7612c7c3973e
I20260810 18:41:59.430282 12843 ts_tablet_manager.cc:1939] T 4ba89426ec8949f4886a7612c7c3973e P 354ebbbe83ef4c9f818c00b38e246819: Deleting consensus metadata
I20260810 18:41:59.432488 12091 catalog_manager.cc:4841] TS 354ebbbe83ef4c9f818c00b38e246819 (127.4.228.194:44775): tablet 4ba89426ec8949f4886a7612c7c3973e (table test-table [id=c56b8f3d451444218f616e4c18ef38b7]) successfully deleted
I20260810 18:41:59.434087 12804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:59.410263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55714 (local address 127.4.228.193:33575)
0810 18:41:59.410527 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:59.410537 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:59.423059 (+ 12522us) server_negotiation.cc:407] Connection header received
0810 18:41:59.423644 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:59.423651 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:59.423726 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:59.423810 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:59.433092 (+ 9282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:59.433099 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:59.433107 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:41:59.433141 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:59.433201 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:41:59.433221 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:59.433353 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:59.433705 (+ 352us) server_negotiation.cc:299] Negotiation successful
0810 18:41:59.433775 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260810 18:41:59.434142 12842 ts_tablet_manager.cc:1918] T 4ba89426ec8949f4886a7612c7c3973e P 15a1d63e41d74551812c0945bca71401: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:41:59.434309 12842 log.cc:1196] T 4ba89426ec8949f4886a7612c7c3973e P 15a1d63e41d74551812c0945bca71401: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-2/wal/wals/4ba89426ec8949f4886a7612c7c3973e
I20260810 18:41:59.435145 12842 ts_tablet_manager.cc:1939] T 4ba89426ec8949f4886a7612c7c3973e P 15a1d63e41d74551812c0945bca71401: Deleting consensus metadata
I20260810 18:41:59.435320 12388 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:41:59.437121 12089 catalog_manager.cc:4841] TS 15a1d63e41d74551812c0945bca71401 (127.4.228.195:46315): tablet 4ba89426ec8949f4886a7612c7c3973e (table test-table [id=c56b8f3d451444218f616e4c18ef38b7]) successfully deleted
I20260810 18:41:59.533607 12841 ts_tablet_manager.cc:1918] T 4ba89426ec8949f4886a7612c7c3973e P e78ca2699da94ca5b1d4d218b72c8a6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:41:59.533962 12841 log.cc:1196] T 4ba89426ec8949f4886a7612c7c3973e P e78ca2699da94ca5b1d4d218b72c8a6d: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387267874959-5011-0/minicluster-data/ts-0/wal/wals/4ba89426ec8949f4886a7612c7c3973e
I20260810 18:41:59.534500 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 12208
W20260810 18:41:59.565326 12089 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.228.193:33575 (error 108)
I20260810 18:41:59.565567 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 12400
I20260810 18:41:59.567605 12803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:59.567081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:33575 (local address 127.0.0.1:55724)
0810 18:41:59.567325 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:59.567455 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.193:33575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260810 18:41:59.567935 12089 catalog_manager.cc:4584] TS e78ca2699da94ca5b1d4d218b72c8a6d (127.4.228.193:33575): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4ba89426ec8949f4886a7612c7c3973e: Network error: Client connection negotiation failed: client connection to 127.4.228.193:33575: connect: Connection refused (error 111)
I20260810 18:41:59.569149 12800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:59.566679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42226 (local address 127.4.228.194:44775)
0810 18:41:59.566864 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:59.566870 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:59.566892 (+ 22us) server_negotiation.cc:407] Connection header received
0810 18:41:59.567100 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:59.567110 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:59.567210 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:59.567349 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:59.568429 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:59.568434 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:59.568438 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:41:59.568455 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:59.568487 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:41:59.568499 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:59.568599 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:59.568761 (+ 162us) server_negotiation.cc:299] Negotiation successful
0810 18:41:59.568812 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 18:41:59.570066 12577 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:41:59.622217 12803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:59.621642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:33575 (local address 127.0.0.1:55730)
0810 18:41:59.621920 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:59.622018 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.193:33575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260810 18:41:59.623477 12803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:59.623069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:33575 (local address 127.0.0.1:55742)
0810 18:41:59.623254 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:59.623348 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.193:33575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260810 18:41:59.661743 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 12400
I20260810 18:41:59.684288 12803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:59.683595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:33575 (local address 127.0.0.1:55748)
0810 18:41:59.683869 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:59.683997 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.193:33575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260810 18:41:59.685592 12803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:59.685134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:33575 (local address 127.0.0.1:55764)
0810 18:41:59.685321 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:59.685422 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.193:33575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 18:41:59.691404 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 12596
I20260810 18:41:59.694962 12799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:59.692329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60014 (local address 127.4.228.195:46315)
0810 18:41:59.692566 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0810 18:41:59.692618 (+ 52us) server_negotiation.cc:399] Waiting for connection header
0810 18:41:59.692644 (+ 26us) server_negotiation.cc:407] Connection header received
0810 18:41:59.692827 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:41:59.692844 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:41:59.692931 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:41:59.693054 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:41:59.694113 (+ 1059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:41:59.694119 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:41:59.694123 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:41:59.694142 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:41:59.694180 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:41:59.694195 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:41:59.694412 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0810 18:41:59.694559 (+ 147us) server_negotiation.cc:299] Negotiation successful
0810 18:41:59.694636 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260810 18:41:59.695788 12770 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:41:59.766741 12803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:59.766076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:33575 (local address 127.0.0.1:55774)
0810 18:41:59.766362 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:59.766538 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.193:33575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260810 18:41:59.768153 12803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:59.767616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:33575 (local address 127.0.0.1:55786)
0810 18:41:59.767869 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:59.767969 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.193:33575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
I20260810 18:41:59.791366 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 12596
I20260810 18:41:59.817878 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 12065
I20260810 18:41:59.819551 12156 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:41:59.941586 12803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:41:59.941008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:33575 (local address 127.0.0.1:55796)
0810 18:41:59.941289 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0810 18:41:59.941377 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.193:33575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260810 18:41:59.941646 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 12065
2026-08-10T18:41:59Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2990 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260810 18:41:59.997447 5011 mini_oidc.cc:152] Starting JWKS server
I20260810 18:42:00.000878 5011 webserver.cc:460] Webserver started at https://127.0.0.1:39519/ using document root <none> and password file <none>
I20260810 18:42:00.001312 5011 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 18:42:00.002207 5011 webserver.cc:460] Webserver started at http://127.0.0.1:37739/ using document root <none> and password file <none>
2026-08-10T18:42:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:42:00Z Disabled control of system clock
I20260810 18:42:00.045495 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:34817
--webserver_interface=127.4.228.254
--webserver_port=0
--builtin_ntp_servers=127.4.228.212:40671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:34817
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39519/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387267874959-5011-0/testchainca.cert with env {}
W20260810 18:42:00.436256 12863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:42:00.436722 12863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:42:00.436909 12863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:42:00.444862 12863 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:42:00.445072 12863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:42:00.445324 12863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:42:00.445508 12863 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:42:00.445597 12863 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 18:42:00.445719 12863 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 18:42:00.445796 12863 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39519/jwks.json
I20260810 18:42:00.457145 12863 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:40671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:34817
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:34817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39519/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387267874959-5011-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:42:00.458899 12863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:42:00.460469 12863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:42:00.475482 12872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:00.475559 12870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:00.475536 12869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:42:00.477625 12863 server_base.cc:1034] running on GCE node
I20260810 18:42:00.479265 12863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:42:00.481158 12863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:42:00.482565 12863 hybrid_clock.cc:648] HybridClock initialized: now 1786387320482423 us; error 94 us; skew 500 ppm
I20260810 18:42:00.483052 12863 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:42:00.485971 12863 webserver.cc:460] Webserver started at http://127.4.228.254:44181/ using document root <none> and password file <none>
I20260810 18:42:00.486891 12863 fs_manager.cc:359] Metadata directory not provided
I20260810 18:42:00.487049 12863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:42:00.487444 12863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:42:00.490845 12863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "2e7b653577384024aa31325e93c4c5b5"
format_stamp: "Formatted at 2026-08-10 18:42:00 on dist-test-slave-7kzw"
server_key: "b388697f4afa501766143a969d95656b"
server_key_iv: "5b6673f7e8849b93d2441df3a2db18b5"
server_key_version: "encryptionkey@0"
I20260810 18:42:00.491782 12863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "2e7b653577384024aa31325e93c4c5b5"
format_stamp: "Formatted at 2026-08-10 18:42:00 on dist-test-slave-7kzw"
server_key: "b388697f4afa501766143a969d95656b"
server_key_iv: "5b6673f7e8849b93d2441df3a2db18b5"
server_key_version: "encryptionkey@0"
I20260810 18:42:00.498661 12863 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260810 18:42:00.503631 12878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:00.505697 12863 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260810 18:42:00.505980 12863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "2e7b653577384024aa31325e93c4c5b5"
format_stamp: "Formatted at 2026-08-10 18:42:00 on dist-test-slave-7kzw"
server_key: "b388697f4afa501766143a969d95656b"
server_key_iv: "5b6673f7e8849b93d2441df3a2db18b5"
server_key_version: "encryptionkey@0"
I20260810 18:42:00.506412 12863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:42:00.536207 12863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:42:00.537228 12863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:42:00.537611 12863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:42:00.559580 12863 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:34817
I20260810 18:42:00.559697 12949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:34817 every 8 connection(s)
I20260810 18:42:00.561152 12863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:42:00.565944 12951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:42:00.566344 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 12863
I20260810 18:42:00.566608 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:42:00.567595 5011 external_mini_cluster.cc:1428] Setting key 99a2435560d07a3d4c3e10bcb7bf4f41
I20260810 18:42:00.581454 12951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e7b653577384024aa31325e93c4c5b5: Bootstrap starting.
I20260810 18:42:00.583599 12955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:00.570186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44216 (local address 127.4.228.254:34817)
0810 18:42:00.571480 (+ 1294us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:00.571494 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:00.571544 (+ 50us) server_negotiation.cc:407] Connection header received
0810 18:42:00.573676 (+ 2132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:00.573706 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:00.574272 (+ 566us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:00.574820 (+ 548us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:00.575468 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:00.576806 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:00.578284 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:00.579128 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:00.579356 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:00.580947 (+ 1591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:00.580956 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:00.580972 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:42:00.581594 (+ 622us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:00.581733 (+ 139us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:42:00.581758 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:00.581898 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:00.582129 (+ 231us) server_negotiation.cc:299] Negotiation successful
0810 18:42:00.582566 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":741,"thread_start_us":462,"threads_started":1}
I20260810 18:42:00.585723 12951 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e7b653577384024aa31325e93c4c5b5: Neither blocks nor log segments found. Creating new log.
I20260810 18:42:00.587208 12951 log.cc:824] T 00000000000000000000000000000000 P 2e7b653577384024aa31325e93c4c5b5: Log is configured to *not* fsync() on all Append() calls
I20260810 18:42:00.588176 5011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:42:00.590634 12951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e7b653577384024aa31325e93c4c5b5: No bootstrap required, opened a new log
I20260810 18:42:00.599223 12951 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e7b653577384024aa31325e93c4c5b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e7b653577384024aa31325e93c4c5b5" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 34817 } }
I20260810 18:42:00.599712 12951 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e7b653577384024aa31325e93c4c5b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:42:00.599862 12951 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e7b653577384024aa31325e93c4c5b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e7b653577384024aa31325e93c4c5b5, State: Initialized, Role: FOLLOWER
I20260810 18:42:00.600575 12951 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e7b653577384024aa31325e93c4c5b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e7b653577384024aa31325e93c4c5b5" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 34817 } }
I20260810 18:42:00.600845 12951 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e7b653577384024aa31325e93c4c5b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:42:00.601042 12951 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e7b653577384024aa31325e93c4c5b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:42:00.601261 12951 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e7b653577384024aa31325e93c4c5b5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:42:00.603894 12951 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e7b653577384024aa31325e93c4c5b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e7b653577384024aa31325e93c4c5b5" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 34817 } }
I20260810 18:42:00.604724 12951 leader_election.cc:302] T 00000000000000000000000000000000 P 2e7b653577384024aa31325e93c4c5b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e7b653577384024aa31325e93c4c5b5; no voters:
I20260810 18:42:00.605773 12951 leader_election.cc:288] T 00000000000000000000000000000000 P 2e7b653577384024aa31325e93c4c5b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:42:00.605968 12961 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e7b653577384024aa31325e93c4c5b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:42:00.606945 12961 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e7b653577384024aa31325e93c4c5b5 [term 1 LEADER]: Becoming Leader. State: Replica: 2e7b653577384024aa31325e93c4c5b5, State: Running, Role: LEADER
I20260810 18:42:00.607718 12961 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e7b653577384024aa31325e93c4c5b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e7b653577384024aa31325e93c4c5b5" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 34817 } }
I20260810 18:42:00.608537 12951 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e7b653577384024aa31325e93c4c5b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:42:00.611308 12962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e7b653577384024aa31325e93c4c5b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e7b653577384024aa31325e93c4c5b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e7b653577384024aa31325e93c4c5b5" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 34817 } } }
I20260810 18:42:00.612039 12962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e7b653577384024aa31325e93c4c5b5 [sys.catalog]: This master's current role is: LEADER
I20260810 18:42:00.611500 12963 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e7b653577384024aa31325e93c4c5b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e7b653577384024aa31325e93c4c5b5. Latest consensus state: current_term: 1 leader_uuid: "2e7b653577384024aa31325e93c4c5b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e7b653577384024aa31325e93c4c5b5" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 34817 } } }
I20260810 18:42:00.613350 12963 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e7b653577384024aa31325e93c4c5b5 [sys.catalog]: This master's current role is: LEADER
I20260810 18:42:00.617897 12968 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:42:00.623383 12968 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:42:00.632987 12968 catalog_manager.cc:1347] Generated new cluster ID: 3db8f1f081ae44cf87939aa88427e306
I20260810 18:42:00.633136 12968 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:42:00.651684 12968 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:42:00.654055 12968 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:42:00.666368 12968 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e7b653577384024aa31325e93c4c5b5: Generated new TSK 0
I20260810 18:42:00.667609 12968 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 18:42:00.719264 12993 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260810 18:42:00.719516 12993 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 18:42:00.721858 12955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:00.689463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44228 (local address 127.4.228.254:34817)
0810 18:42:00.689764 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:00.689773 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:00.690214 (+ 441us) server_negotiation.cc:407] Connection header received
0810 18:42:00.690743 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:00.690747 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:00.690837 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:00.690959 (+ 122us) server_negotiation.cc:226] Negotiated authn=JWT
0810 18:42:00.691745 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:00.692537 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:00.694721 (+ 2184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:00.695192 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:00.695302 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:00.696082 (+ 780us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 18:42:00.721184 (+ 25102us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39519/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 18:42:00.721565 (+ 381us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44228: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39519/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":85}
W20260810 18:42:00.722039 12955 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44228: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39519/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260810 18:42:00.724372 12991 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:00.689333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:34817 (local address 127.0.0.1:44228)
0810 18:42:00.690082 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:00.690104 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:00.690324 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:00.691037 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:00.691043 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:00.691057 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0810 18:42:00.691506 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:00.691521 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:00.692705 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:00.692711 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:00.694520 (+ 1809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:00.694540 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:00.695388 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:00.695395 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:00.695614 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:00.695726 (+ 112us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 18:42:00.721572 (+ 25846us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39519/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 18:42:00.721990 (+ 418us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.228.254:34817: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39519/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":359,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.4.228.254:34817: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39519/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260810 18:42:00.726778 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 12863
I20260810 18:42:00.728581 12945 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:42:00.826866 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 12863
2026-08-10T18:42:00Z chronyd exiting
I20260810 18:42:00.888723 5011 test_util.cc:175] -----------------------------------------------
I20260810 18:42:00.888880 5011 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387267874959-5011-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (896 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260810 18:42:00.894189 5011 mini_oidc.cc:152] Starting JWKS server
I20260810 18:42:00.896293 5011 webserver.cc:460] Webserver started at https://127.0.0.1:41647/ using document root <none> and password file <none>
I20260810 18:42:00.896795 5011 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 18:42:00.897619 5011 webserver.cc:460] Webserver started at http://127.0.0.1:42037/ using document root <none> and password file <none>
2026-08-10T18:42:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:42:00Z Disabled control of system clock
I20260810 18:42:00.940357 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:39155
--webserver_interface=127.4.228.254
--webserver_port=0
--builtin_ntp_servers=127.4.228.212:40375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:39155
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41647/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387267874959-5011-0/testchainca.cert with env {}
W20260810 18:42:01.304370 13006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:42:01.304729 13006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:42:01.304899 13006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:42:01.312501 13006 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:42:01.312631 13006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:42:01.312677 13006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:42:01.312798 13006 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:42:01.312862 13006 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 18:42:01.312950 13006 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 18:42:01.313011 13006 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41647/jwks.json
I20260810 18:42:01.324128 13006 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:40375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:39155
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:39155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41647/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387267874959-5011-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:42:01.325662 13006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:42:01.327131 13006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:42:01.340179 13012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:01.340232 13015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:01.340195 13013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:42:01.341385 13006 server_base.cc:1034] running on GCE node
I20260810 18:42:01.342504 13006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:42:01.344206 13006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:42:01.345436 13006 hybrid_clock.cc:648] HybridClock initialized: now 1786387321345368 us; error 61 us; skew 500 ppm
I20260810 18:42:01.345896 13006 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:42:01.352372 13006 webserver.cc:460] Webserver started at http://127.4.228.254:32855/ using document root <none> and password file <none>
I20260810 18:42:01.353196 13006 fs_manager.cc:359] Metadata directory not provided
I20260810 18:42:01.353340 13006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:42:01.353726 13006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:42:01.356976 13006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "75c759ff459f4137baf6f3eb6f608107"
format_stamp: "Formatted at 2026-08-10 18:42:01 on dist-test-slave-7kzw"
server_key: "19206bdeb484aefd66f13b3998617e0a"
server_key_iv: "01850f31b5b7d0b6525a346f20391552"
server_key_version: "encryptionkey@0"
I20260810 18:42:01.357952 13006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "75c759ff459f4137baf6f3eb6f608107"
format_stamp: "Formatted at 2026-08-10 18:42:01 on dist-test-slave-7kzw"
server_key: "19206bdeb484aefd66f13b3998617e0a"
server_key_iv: "01850f31b5b7d0b6525a346f20391552"
server_key_version: "encryptionkey@0"
I20260810 18:42:01.363741 13006 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 18:42:01.367478 13026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:01.369112 13006 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:42:01.369279 13006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "75c759ff459f4137baf6f3eb6f608107"
format_stamp: "Formatted at 2026-08-10 18:42:01 on dist-test-slave-7kzw"
server_key: "19206bdeb484aefd66f13b3998617e0a"
server_key_iv: "01850f31b5b7d0b6525a346f20391552"
server_key_version: "encryptionkey@0"
I20260810 18:42:01.369479 13006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:42:01.393558 13006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:42:01.394483 13006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:42:01.394775 13006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:42:01.416328 13006 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:39155
I20260810 18:42:01.416371 13099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:39155 every 8 connection(s)
I20260810 18:42:01.418524 13006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:42:01.419453 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 13006
I20260810 18:42:01.419739 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:42:01.420650 5011 external_mini_cluster.cc:1428] Setting key 330a41f49eae84d74cdb1113b24b5420
I20260810 18:42:01.424436 13100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:42:01.439096 13103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:01.423028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59698 (local address 127.4.228.254:39155)
0810 18:42:01.424305 (+ 1277us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:01.424318 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:01.424364 (+ 46us) server_negotiation.cc:407] Connection header received
0810 18:42:01.426047 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:01.426071 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:01.426675 (+ 604us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:01.427078 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:01.427739 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:01.430072 (+ 2333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:01.432447 (+ 2375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:01.433520 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:01.433793 (+ 273us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:01.435649 (+ 1856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:01.435662 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:01.435684 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:42:01.436679 (+ 995us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:01.436794 (+ 115us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:42:01.436826 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:01.436986 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:01.437223 (+ 237us) server_negotiation.cc:299] Negotiation successful
0810 18:42:01.437624 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":748,"thread_start_us":464,"threads_started":1}
I20260810 18:42:01.443523 13100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75c759ff459f4137baf6f3eb6f608107: Bootstrap starting.
I20260810 18:42:01.444680 5011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:42:01.448357 13100 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75c759ff459f4137baf6f3eb6f608107: Neither blocks nor log segments found. Creating new log.
I20260810 18:42:01.450062 13100 log.cc:824] T 00000000000000000000000000000000 P 75c759ff459f4137baf6f3eb6f608107: Log is configured to *not* fsync() on all Append() calls
I20260810 18:42:01.455574 13100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75c759ff459f4137baf6f3eb6f608107: No bootstrap required, opened a new log
I20260810 18:42:01.467363 13100 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75c759ff459f4137baf6f3eb6f608107 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75c759ff459f4137baf6f3eb6f608107" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 39155 } }
I20260810 18:42:01.467852 13100 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75c759ff459f4137baf6f3eb6f608107 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:42:01.467985 13100 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75c759ff459f4137baf6f3eb6f608107 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75c759ff459f4137baf6f3eb6f608107, State: Initialized, Role: FOLLOWER
I20260810 18:42:01.468720 13100 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75c759ff459f4137baf6f3eb6f608107 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75c759ff459f4137baf6f3eb6f608107" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 39155 } }
I20260810 18:42:01.469010 13100 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75c759ff459f4137baf6f3eb6f608107 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:42:01.469214 13100 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75c759ff459f4137baf6f3eb6f608107 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:42:01.469439 13100 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75c759ff459f4137baf6f3eb6f608107 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:42:01.472115 13100 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75c759ff459f4137baf6f3eb6f608107 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75c759ff459f4137baf6f3eb6f608107" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 39155 } }
I20260810 18:42:01.472767 13100 leader_election.cc:302] T 00000000000000000000000000000000 P 75c759ff459f4137baf6f3eb6f608107 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75c759ff459f4137baf6f3eb6f608107; no voters:
I20260810 18:42:01.473600 13100 leader_election.cc:288] T 00000000000000000000000000000000 P 75c759ff459f4137baf6f3eb6f608107 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:42:01.473783 13105 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75c759ff459f4137baf6f3eb6f608107 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:42:01.474857 13105 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75c759ff459f4137baf6f3eb6f608107 [term 1 LEADER]: Becoming Leader. State: Replica: 75c759ff459f4137baf6f3eb6f608107, State: Running, Role: LEADER
I20260810 18:42:01.475595 13105 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75c759ff459f4137baf6f3eb6f608107 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75c759ff459f4137baf6f3eb6f608107" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 39155 } }
I20260810 18:42:01.476665 13100 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75c759ff459f4137baf6f3eb6f608107 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:42:01.479530 13108 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75c759ff459f4137baf6f3eb6f608107 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75c759ff459f4137baf6f3eb6f608107. Latest consensus state: current_term: 1 leader_uuid: "75c759ff459f4137baf6f3eb6f608107" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75c759ff459f4137baf6f3eb6f608107" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 39155 } } }
I20260810 18:42:01.480051 13108 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75c759ff459f4137baf6f3eb6f608107 [sys.catalog]: This master's current role is: LEADER
I20260810 18:42:01.480630 13107 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75c759ff459f4137baf6f3eb6f608107 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "75c759ff459f4137baf6f3eb6f608107" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75c759ff459f4137baf6f3eb6f608107" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 39155 } } }
I20260810 18:42:01.481027 13107 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75c759ff459f4137baf6f3eb6f608107 [sys.catalog]: This master's current role is: LEADER
I20260810 18:42:01.483732 13113 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:42:01.489940 13113 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:42:01.500267 13113 catalog_manager.cc:1347] Generated new cluster ID: 9a5a77b82c5c45c39c7b690d4f58e3eb
I20260810 18:42:01.500407 13113 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:42:01.522562 13113 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:42:01.524626 13113 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:42:01.539229 13113 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 75c759ff459f4137baf6f3eb6f608107: Generated new TSK 0
I20260810 18:42:01.540486 13113 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 18:42:01.577574 13134 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260810 18:42:01.577739 13134 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 18:42:01.579030 13103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:01.557811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59706 (local address 127.4.228.254:39155)
0810 18:42:01.558047 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:01.558053 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:01.558391 (+ 338us) server_negotiation.cc:407] Connection header received
0810 18:42:01.558689 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:01.558694 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:01.558789 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:01.558944 (+ 155us) server_negotiation.cc:226] Negotiated authn=JWT
0810 18:42:01.559560 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:01.560475 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:01.562635 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:01.563160 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:01.563271 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:01.563954 (+ 683us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 18:42:01.578382 (+ 14428us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41647/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 18:42:01.578775 (+ 393us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59706: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41647/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":56}
W20260810 18:42:01.579195 13103 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59706: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41647/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260810 18:42:01.579450 13133 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:01.557460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:39155 (local address 127.0.0.1:59706)
0810 18:42:01.558202 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:01.558224 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:01.558404 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:01.558994 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:01.558999 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:01.559016 (+ 17us) client_negotiation.cc:189] Negotiated authn=JWT
0810 18:42:01.559349 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:01.559364 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:01.560640 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:01.560646 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:01.562425 (+ 1779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:01.562442 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:01.563375 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:01.563381 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:01.563598 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:01.563649 (+ 51us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 18:42:01.578696 (+ 15047us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41647/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 18:42:01.579013 (+ 317us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.228.254:39155: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41647/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":583,"threads_started":1}
I20260810 18:42:01.580641 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 13006
I20260810 18:42:01.582171 13094 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:42:01.681456 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 13006
2026-08-10T18:42:01Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (856 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260810 18:42:01.749953 5011 mini_oidc.cc:152] Starting JWKS server
I20260810 18:42:01.752096 5011 webserver.cc:460] Webserver started at https://127.0.0.1:36513/ using document root <none> and password file <none>
I20260810 18:42:01.752364 5011 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 18:42:01.753123 5011 webserver.cc:460] Webserver started at http://127.0.0.1:45047/ using document root <none> and password file <none>
2026-08-10T18:42:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:42:01Z Disabled control of system clock
I20260810 18:42:01.792588 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:46341
--webserver_interface=127.4.228.254
--webserver_port=0
--builtin_ntp_servers=127.4.228.212:41483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:46341
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36513/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 18:42:02.163686 13145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:42:02.164093 13145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:42:02.164264 13145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:42:02.171869 13145 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:42:02.172039 13145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:42:02.172138 13145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:42:02.172260 13145 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:42:02.172345 13145 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 18:42:02.172438 13145 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 18:42:02.172518 13145 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36513/jwks.json
I20260810 18:42:02.183079 13145 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:41483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:46341
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:46341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36513/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:42:02.184659 13145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:42:02.186769 13145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:42:02.199873 13160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:02.199839 13164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:02.199779 13161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:42:02.200923 13145 server_base.cc:1034] running on GCE node
I20260810 18:42:02.201886 13145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:42:02.203573 13145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:42:02.204846 13145 hybrid_clock.cc:648] HybridClock initialized: now 1786387322204776 us; error 72 us; skew 500 ppm
I20260810 18:42:02.205292 13145 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:42:02.208127 13145 webserver.cc:460] Webserver started at http://127.4.228.254:33679/ using document root <none> and password file <none>
I20260810 18:42:02.208916 13145 fs_manager.cc:359] Metadata directory not provided
I20260810 18:42:02.209059 13145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:42:02.209419 13145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:42:02.212616 13145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "8370659969bd415aa92565952c1a9de8"
format_stamp: "Formatted at 2026-08-10 18:42:02 on dist-test-slave-7kzw"
server_key: "c7e8fae2625404d59153b555bde27cc5"
server_key_iv: "833d9249da718c53a5c689eb449ec9b0"
server_key_version: "encryptionkey@0"
I20260810 18:42:02.213480 13145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "8370659969bd415aa92565952c1a9de8"
format_stamp: "Formatted at 2026-08-10 18:42:02 on dist-test-slave-7kzw"
server_key: "c7e8fae2625404d59153b555bde27cc5"
server_key_iv: "833d9249da718c53a5c689eb449ec9b0"
server_key_version: "encryptionkey@0"
I20260810 18:42:02.219265 13145 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 18:42:02.223404 13171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:02.224941 13145 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:42:02.225201 13145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "8370659969bd415aa92565952c1a9de8"
format_stamp: "Formatted at 2026-08-10 18:42:02 on dist-test-slave-7kzw"
server_key: "c7e8fae2625404d59153b555bde27cc5"
server_key_iv: "833d9249da718c53a5c689eb449ec9b0"
server_key_version: "encryptionkey@0"
I20260810 18:42:02.225461 13145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:42:02.249274 13145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:42:02.250250 13145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:42:02.250664 13145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:42:02.271098 13145 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:46341
I20260810 18:42:02.271106 13244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:46341 every 8 connection(s)
I20260810 18:42:02.272599 13145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:42:02.276901 13245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:42:02.279666 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 13145
I20260810 18:42:02.279878 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:42:02.280715 5011 external_mini_cluster.cc:1428] Setting key edc2d0c8487e2effbb799f7f97c856ef
I20260810 18:42:02.290202 13245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8370659969bd415aa92565952c1a9de8: Bootstrap starting.
I20260810 18:42:02.293720 13245 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8370659969bd415aa92565952c1a9de8: Neither blocks nor log segments found. Creating new log.
I20260810 18:42:02.295076 13245 log.cc:824] T 00000000000000000000000000000000 P 8370659969bd415aa92565952c1a9de8: Log is configured to *not* fsync() on all Append() calls
I20260810 18:42:02.295506 13250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:02.283116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51218 (local address 127.4.228.254:46341)
0810 18:42:02.284352 (+ 1236us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:02.284365 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:02.284411 (+ 46us) server_negotiation.cc:407] Connection header received
0810 18:42:02.286124 (+ 1713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:02.286150 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:02.286797 (+ 647us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:02.287240 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:02.287919 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:02.289231 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:02.290750 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:02.291472 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:02.291706 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:02.292986 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:02.292994 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:02.293010 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:42:02.293562 (+ 552us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:02.293670 (+ 108us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:42:02.293693 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:02.293847 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:02.294046 (+ 199us) server_negotiation.cc:299] Negotiation successful
0810 18:42:02.294390 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":759,"thread_start_us":466,"threads_started":1}
I20260810 18:42:02.299084 13245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8370659969bd415aa92565952c1a9de8: No bootstrap required, opened a new log
I20260810 18:42:02.300112 5011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:42:02.308209 13245 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8370659969bd415aa92565952c1a9de8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8370659969bd415aa92565952c1a9de8" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46341 } }
I20260810 18:42:02.308727 13245 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8370659969bd415aa92565952c1a9de8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:42:02.308861 13245 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8370659969bd415aa92565952c1a9de8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8370659969bd415aa92565952c1a9de8, State: Initialized, Role: FOLLOWER
I20260810 18:42:02.309494 13245 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8370659969bd415aa92565952c1a9de8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8370659969bd415aa92565952c1a9de8" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46341 } }
I20260810 18:42:02.309780 13245 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8370659969bd415aa92565952c1a9de8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:42:02.309940 13245 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8370659969bd415aa92565952c1a9de8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:42:02.310200 13245 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8370659969bd415aa92565952c1a9de8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:42:02.312692 13245 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8370659969bd415aa92565952c1a9de8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8370659969bd415aa92565952c1a9de8" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46341 } }
I20260810 18:42:02.313212 13245 leader_election.cc:302] T 00000000000000000000000000000000 P 8370659969bd415aa92565952c1a9de8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8370659969bd415aa92565952c1a9de8; no voters:
I20260810 18:42:02.313804 13245 leader_election.cc:288] T 00000000000000000000000000000000 P 8370659969bd415aa92565952c1a9de8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:42:02.314018 13252 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8370659969bd415aa92565952c1a9de8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:42:02.314915 13252 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8370659969bd415aa92565952c1a9de8 [term 1 LEADER]: Becoming Leader. State: Replica: 8370659969bd415aa92565952c1a9de8, State: Running, Role: LEADER
I20260810 18:42:02.315657 13252 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8370659969bd415aa92565952c1a9de8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8370659969bd415aa92565952c1a9de8" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46341 } }
I20260810 18:42:02.316385 13245 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8370659969bd415aa92565952c1a9de8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:42:02.318949 13254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8370659969bd415aa92565952c1a9de8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8370659969bd415aa92565952c1a9de8. Latest consensus state: current_term: 1 leader_uuid: "8370659969bd415aa92565952c1a9de8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8370659969bd415aa92565952c1a9de8" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46341 } } }
I20260810 18:42:02.319448 13254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8370659969bd415aa92565952c1a9de8 [sys.catalog]: This master's current role is: LEADER
I20260810 18:42:02.319327 13253 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8370659969bd415aa92565952c1a9de8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8370659969bd415aa92565952c1a9de8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8370659969bd415aa92565952c1a9de8" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46341 } } }
I20260810 18:42:02.319705 13253 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8370659969bd415aa92565952c1a9de8 [sys.catalog]: This master's current role is: LEADER
I20260810 18:42:02.321836 13268 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:42:02.328281 13268 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:42:02.339005 13268 catalog_manager.cc:1347] Generated new cluster ID: 582e8cb3384145e8bd2608410385ad50
I20260810 18:42:02.339191 13268 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:42:02.378808 13268 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:42:02.380221 13268 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:42:02.395790 13268 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8370659969bd415aa92565952c1a9de8: Generated new TSK 0
I20260810 18:42:02.396790 13268 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:42:02.431172 13250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:02.421038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51228 (local address 127.4.228.254:46341)
0810 18:42:02.421300 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:02.421308 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:02.421932 (+ 624us) server_negotiation.cc:407] Connection header received
0810 18:42:02.422176 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:02.422184 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:02.422300 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:02.422533 (+ 233us) server_negotiation.cc:226] Negotiated authn=JWT
0810 18:42:02.423042 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:02.424338 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:02.425965 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:02.426793 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:02.426936 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:02.427553 (+ 617us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 18:42:02.430380 (+ 2827us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36513/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 18:42:02.430788 (+ 408us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51228: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36513/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":86}
W20260810 18:42:02.431330 13250 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51228: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36513/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260810 18:42:02.433104 13290 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:02.421088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:46341 (local address 127.0.0.1:51228)
0810 18:42:02.421822 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:02.421839 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:02.421970 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:02.422567 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:02.422571 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:02.422582 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0810 18:42:02.422862 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:02.422872 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:02.424524 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:02.424530 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:02.425793 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:02.425809 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:02.427021 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:02.427027 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:02.427172 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:02.427209 (+ 37us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 18:42:02.430663 (+ 3454us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36513/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 18:42:02.432672 (+ 2009us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.228.254:46341: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36513/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":579,"mutex_wait_us":1638,"thread_start_us":414,"threads_started":1}
I20260810 18:42:02.434302 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 13145
I20260810 18:42:02.435773 13240 generic_service.cc:196] Checking for leaks (request via RPC)
E20260810 18:42:02.445974 13291 webserver.cc:533] Webserver: sslize failed:
E20260810 18:42:02.446161 13291 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 18:42:02.536999 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 13145
2026-08-10T18:42:02Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (854 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-10T18:42:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:42:02Z Disabled control of system clock
I20260810 18:42:02.644294 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:34707
--webserver_interface=127.4.228.254
--webserver_port=0
--builtin_ntp_servers=127.4.228.212:42705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:34707
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387267874959-5011-0/testchainca.cert with env {}
W20260810 18:42:03.007663 13297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:42:03.008042 13297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:42:03.008136 13297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:42:03.015797 13297 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:42:03.015990 13297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:42:03.016039 13297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:42:03.016109 13297 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:42:03.016167 13297 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 18:42:03.016234 13297 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 18:42:03.016273 13297 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260810 18:42:03.028502 13297 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:42705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:34707
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:34707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387267874959-5011-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:42:03.030067 13297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:42:03.031597 13297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:42:03.044807 13303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:03.044878 13310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:03.044878 13304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:42:03.045745 13297 server_base.cc:1034] running on GCE node
I20260810 18:42:03.046727 13297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:42:03.048276 13297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:42:03.049574 13297 hybrid_clock.cc:648] HybridClock initialized: now 1786387323049527 us; error 52 us; skew 500 ppm
I20260810 18:42:03.050040 13297 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:42:03.052922 13297 webserver.cc:460] Webserver started at http://127.4.228.254:36657/ using document root <none> and password file <none>
I20260810 18:42:03.053824 13297 fs_manager.cc:359] Metadata directory not provided
I20260810 18:42:03.053987 13297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:42:03.054381 13297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:42:03.057746 13297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "7479eb40c0ed4dfb9dae2e4bf9440cfd"
format_stamp: "Formatted at 2026-08-10 18:42:03 on dist-test-slave-7kzw"
server_key: "96ae11d75b677eb57a1b2619d1ec2e17"
server_key_iv: "1b4fb7d82ec45b1ce1e0e78a90b08c43"
server_key_version: "encryptionkey@0"
I20260810 18:42:03.058811 13297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "7479eb40c0ed4dfb9dae2e4bf9440cfd"
format_stamp: "Formatted at 2026-08-10 18:42:03 on dist-test-slave-7kzw"
server_key: "96ae11d75b677eb57a1b2619d1ec2e17"
server_key_iv: "1b4fb7d82ec45b1ce1e0e78a90b08c43"
server_key_version: "encryptionkey@0"
I20260810 18:42:03.065114 13297 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 18:42:03.069350 13317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:03.071120 13297 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 18:42:03.071377 13297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "7479eb40c0ed4dfb9dae2e4bf9440cfd"
format_stamp: "Formatted at 2026-08-10 18:42:03 on dist-test-slave-7kzw"
server_key: "96ae11d75b677eb57a1b2619d1ec2e17"
server_key_iv: "1b4fb7d82ec45b1ce1e0e78a90b08c43"
server_key_version: "encryptionkey@0"
I20260810 18:42:03.071667 13297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:42:03.108712 13297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:42:03.109838 13297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:42:03.110195 13297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:42:03.133049 13297 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:34707
I20260810 18:42:03.133056 13402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:34707 every 8 connection(s)
I20260810 18:42:03.134716 13297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:42:03.139221 13404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:42:03.144539 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 13297
I20260810 18:42:03.144838 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:42:03.145532 5011 external_mini_cluster.cc:1428] Setting key bc843bfd714d549f50310c33fbc6043d
I20260810 18:42:03.151265 13404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7479eb40c0ed4dfb9dae2e4bf9440cfd: Bootstrap starting.
I20260810 18:42:03.155346 13404 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7479eb40c0ed4dfb9dae2e4bf9440cfd: Neither blocks nor log segments found. Creating new log.
I20260810 18:42:03.156728 13404 log.cc:824] T 00000000000000000000000000000000 P 7479eb40c0ed4dfb9dae2e4bf9440cfd: Log is configured to *not* fsync() on all Append() calls
I20260810 18:42:03.160751 13404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7479eb40c0ed4dfb9dae2e4bf9440cfd: No bootstrap required, opened a new log
I20260810 18:42:03.162037 13410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:03.147983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39606 (local address 127.4.228.254:34707)
0810 18:42:03.149209 (+ 1226us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:03.149222 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:03.149281 (+ 59us) server_negotiation.cc:407] Connection header received
0810 18:42:03.151123 (+ 1842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:03.151151 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:03.151758 (+ 607us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:03.152167 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:03.152934 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:03.154345 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:03.156378 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:03.157217 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:03.157493 (+ 276us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:03.158795 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:03.158807 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:03.158828 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:42:03.159583 (+ 755us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:03.159715 (+ 132us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:42:03.159750 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:03.159883 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:03.160224 (+ 341us) server_negotiation.cc:299] Negotiation successful
0810 18:42:03.160591 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":710,"thread_start_us":443,"threads_started":1}
I20260810 18:42:03.166786 5011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:42:03.170017 13404 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7479eb40c0ed4dfb9dae2e4bf9440cfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7479eb40c0ed4dfb9dae2e4bf9440cfd" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 34707 } }
I20260810 18:42:03.170572 13404 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7479eb40c0ed4dfb9dae2e4bf9440cfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:42:03.170698 13404 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7479eb40c0ed4dfb9dae2e4bf9440cfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7479eb40c0ed4dfb9dae2e4bf9440cfd, State: Initialized, Role: FOLLOWER
I20260810 18:42:03.171415 13404 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7479eb40c0ed4dfb9dae2e4bf9440cfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7479eb40c0ed4dfb9dae2e4bf9440cfd" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 34707 } }
I20260810 18:42:03.171674 13404 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7479eb40c0ed4dfb9dae2e4bf9440cfd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:42:03.171840 13404 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7479eb40c0ed4dfb9dae2e4bf9440cfd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:42:03.172040 13404 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7479eb40c0ed4dfb9dae2e4bf9440cfd [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:42:03.174552 13404 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7479eb40c0ed4dfb9dae2e4bf9440cfd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7479eb40c0ed4dfb9dae2e4bf9440cfd" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 34707 } }
I20260810 18:42:03.175278 13404 leader_election.cc:302] T 00000000000000000000000000000000 P 7479eb40c0ed4dfb9dae2e4bf9440cfd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7479eb40c0ed4dfb9dae2e4bf9440cfd; no voters:
I20260810 18:42:03.176066 13404 leader_election.cc:288] T 00000000000000000000000000000000 P 7479eb40c0ed4dfb9dae2e4bf9440cfd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:42:03.176266 13412 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7479eb40c0ed4dfb9dae2e4bf9440cfd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:42:03.177374 13412 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7479eb40c0ed4dfb9dae2e4bf9440cfd [term 1 LEADER]: Becoming Leader. State: Replica: 7479eb40c0ed4dfb9dae2e4bf9440cfd, State: Running, Role: LEADER
I20260810 18:42:03.178053 13412 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7479eb40c0ed4dfb9dae2e4bf9440cfd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7479eb40c0ed4dfb9dae2e4bf9440cfd" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 34707 } }
I20260810 18:42:03.178805 13404 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7479eb40c0ed4dfb9dae2e4bf9440cfd [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:42:03.181250 13414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7479eb40c0ed4dfb9dae2e4bf9440cfd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7479eb40c0ed4dfb9dae2e4bf9440cfd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7479eb40c0ed4dfb9dae2e4bf9440cfd" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 34707 } } }
I20260810 18:42:03.181854 13414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7479eb40c0ed4dfb9dae2e4bf9440cfd [sys.catalog]: This master's current role is: LEADER
I20260810 18:42:03.183478 13415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7479eb40c0ed4dfb9dae2e4bf9440cfd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7479eb40c0ed4dfb9dae2e4bf9440cfd. Latest consensus state: current_term: 1 leader_uuid: "7479eb40c0ed4dfb9dae2e4bf9440cfd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7479eb40c0ed4dfb9dae2e4bf9440cfd" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 34707 } } }
I20260810 18:42:03.183946 13415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7479eb40c0ed4dfb9dae2e4bf9440cfd [sys.catalog]: This master's current role is: LEADER
I20260810 18:42:03.186244 13423 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:42:03.191362 13423 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:42:03.202049 13423 catalog_manager.cc:1347] Generated new cluster ID: ec27deee8c6c4ee891726682ed4a8a60
I20260810 18:42:03.202209 13423 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:42:03.212807 13423 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:42:03.214974 13423 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:42:03.234157 13423 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7479eb40c0ed4dfb9dae2e4bf9440cfd: Generated new TSK 0
I20260810 18:42:03.235496 13423 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:42:03.751367 13410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:03.258149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39622 (local address 127.4.228.254:34707)
0810 18:42:03.258437 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:03.258445 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:03.258897 (+ 452us) server_negotiation.cc:407] Connection header received
0810 18:42:03.259191 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:03.259196 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:03.259278 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:03.259421 (+ 143us) server_negotiation.cc:226] Negotiated authn=JWT
0810 18:42:03.260064 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:03.260857 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:03.263165 (+ 2308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:03.263659 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:03.263778 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:03.264486 (+ 708us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 18:42:03.750730 (+486244us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 18:42:03.751095 (+ 365us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39622: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":85}
W20260810 18:42:03.751533 13410 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39622: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260810 18:42:03.751854 13445 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:03.258019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:34707 (local address 127.0.0.1:39622)
0810 18:42:03.258770 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:03.258793 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:03.258965 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:03.259473 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:03.259478 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:03.259493 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0810 18:42:03.259843 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:03.259858 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:03.261017 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:03.261024 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:03.262961 (+ 1937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:03.262981 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:03.263864 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:03.263871 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:03.264076 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:03.264130 (+ 54us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 18:42:03.751072 (+486942us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 18:42:03.751384 (+ 312us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.228.254:34707: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":399,"threads_started":1}
I20260810 18:42:03.753834 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 13297
I20260810 18:42:03.755257 13396 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:42:03.862720 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 13297
2026-08-10T18:42:03Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1310 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (466 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (1708 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:42:06 dist-test-slave-7kzw krb5kdc[13471](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:42:06 dist-test-slave-7kzw krb5kdc[13471](info): set up 2 sockets
Aug 10 18:42:06 dist-test-slave-7kzw krb5kdc[13471](info): commencing operation
krb5kdc: starting...
W20260810 18:42:07.465519 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.374s user 0.005s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:42:07 dist-test-slave-7kzw krb5kdc[13471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:42:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:42:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.228.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
I20260810 18:42:07.755463 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:46603
--webserver_interface=127.4.228.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:41467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:46603
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:42:08.116971 13489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:42:08.117307 13489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:42:08.117476 13489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:42:08.125149 13489 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:42:08.125284 13489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:42:08.125379 13489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:42:08.125479 13489 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:42:08.125568 13489 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:42:08.136170 13489 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:41467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:46603
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:46603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:42:08.137698 13489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:42:08.139256 13489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:42:08.153090 13500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:08.153190 13499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:08.153190 13503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:42:08.153868 13489 server_base.cc:1034] running on GCE node
I20260810 18:42:08.154946 13489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:42:08.156644 13489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:42:08.157968 13489 hybrid_clock.cc:648] HybridClock initialized: now 1786387328157886 us; error 67 us; skew 500 ppm
I20260810 18:42:08.158399 13489 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:42:08 dist-test-slave-7kzw krb5kdc[13471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387328, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:42:08.165697 13489 init.cc:377] Logged in from keytab as kudu/127.4.228.254@KRBTEST.COM (short username kudu)
I20260810 18:42:08.167551 13489 webserver.cc:460] Webserver started at http://127.4.228.254:42169/ using document root <none> and password file <none>
I20260810 18:42:08.168386 13489 fs_manager.cc:359] Metadata directory not provided
I20260810 18:42:08.168555 13489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:42:08.168910 13489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:42:08.172338 13489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "1acdbab8d93242a497efedb13ca98044"
format_stamp: "Formatted at 2026-08-10 18:42:08 on dist-test-slave-7kzw"
server_key: "c5b0e7bd7857b6ff67b30b89daae8a31"
server_key_iv: "8abca378467d445a42c5b9124f976708"
server_key_version: "encryptionkey@0"
I20260810 18:42:08.173264 13489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "1acdbab8d93242a497efedb13ca98044"
format_stamp: "Formatted at 2026-08-10 18:42:08 on dist-test-slave-7kzw"
server_key: "c5b0e7bd7857b6ff67b30b89daae8a31"
server_key_iv: "8abca378467d445a42c5b9124f976708"
server_key_version: "encryptionkey@0"
I20260810 18:42:08.179868 13489 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260810 18:42:08.184537 13512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:08.186350 13489 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260810 18:42:08.186647 13489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "1acdbab8d93242a497efedb13ca98044"
format_stamp: "Formatted at 2026-08-10 18:42:08 on dist-test-slave-7kzw"
server_key: "c5b0e7bd7857b6ff67b30b89daae8a31"
server_key_iv: "8abca378467d445a42c5b9124f976708"
server_key_version: "encryptionkey@0"
I20260810 18:42:08.186901 13489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:42:08.213163 13489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:42:08.218364 13489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:42:08.218748 13489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:42:08.240244 13489 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:46603
I20260810 18:42:08.240280 13592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:46603 every 8 connection(s)
I20260810 18:42:08.241899 13489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:42:08.245535 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 13489
I20260810 18:42:08.245815 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:42:08.246599 5011 external_mini_cluster.cc:1428] Setting key ef9acd97527d9cd54d9921a3f084a01b
I20260810 18:42:08.246922 13594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:42:08.264390 13594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1acdbab8d93242a497efedb13ca98044: Bootstrap starting.
Aug 10 18:42:08 dist-test-slave-7kzw krb5kdc[13471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:42:08.269207 13594 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1acdbab8d93242a497efedb13ca98044: Neither blocks nor log segments found. Creating new log.
I20260810 18:42:08.270979 13594 log.cc:824] T 00000000000000000000000000000000 P 1acdbab8d93242a497efedb13ca98044: Log is configured to *not* fsync() on all Append() calls
I20260810 18:42:08.276007 13594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1acdbab8d93242a497efedb13ca98044: No bootstrap required, opened a new log
I20260810 18:42:08.278329 13598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:08.249061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43244 (local address 127.4.228.254:46603)
0810 18:42:08.250061 (+ 1000us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:08.250079 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:08.250161 (+ 82us) server_negotiation.cc:407] Connection header received
0810 18:42:08.252850 (+ 2689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:08.252894 (+ 44us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:08.253604 (+ 710us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:08.254211 (+ 607us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:08.256158 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:08.258348 (+ 2190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:08.259990 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:08.261412 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:08.261653 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:08.266661 (+ 5008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:08.266678 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:08.266706 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:08.266784 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:08.271899 (+ 5115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:08.273261 (+ 1362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:08.273298 (+ 37us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:08.273310 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:08.273571 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:08.274117 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:08.274127 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:08.274134 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:08.275072 (+ 938us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:08.275451 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:08.275941 (+ 490us) server_negotiation.cc:299] Negotiation successful
0810 18:42:08.276640 (+ 699us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":691,"thread_start_us":400,"threads_started":1}
I20260810 18:42:08.287062 13594 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1acdbab8d93242a497efedb13ca98044 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1acdbab8d93242a497efedb13ca98044" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46603 } }
I20260810 18:42:08.287568 13594 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1acdbab8d93242a497efedb13ca98044 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:42:08.287664 13594 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1acdbab8d93242a497efedb13ca98044 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1acdbab8d93242a497efedb13ca98044, State: Initialized, Role: FOLLOWER
I20260810 18:42:08.288337 13594 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1acdbab8d93242a497efedb13ca98044 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1acdbab8d93242a497efedb13ca98044" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46603 } }
I20260810 18:42:08.288589 13594 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1acdbab8d93242a497efedb13ca98044 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:42:08.288712 13594 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1acdbab8d93242a497efedb13ca98044 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:42:08.288859 13594 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1acdbab8d93242a497efedb13ca98044 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:42:08.291541 13594 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1acdbab8d93242a497efedb13ca98044 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1acdbab8d93242a497efedb13ca98044" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46603 } }
I20260810 18:42:08.292093 13594 leader_election.cc:302] T 00000000000000000000000000000000 P 1acdbab8d93242a497efedb13ca98044 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1acdbab8d93242a497efedb13ca98044; no voters:
I20260810 18:42:08.292891 13594 leader_election.cc:288] T 00000000000000000000000000000000 P 1acdbab8d93242a497efedb13ca98044 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:42:08.293044 13601 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1acdbab8d93242a497efedb13ca98044 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:42:08.294147 13601 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1acdbab8d93242a497efedb13ca98044 [term 1 LEADER]: Becoming Leader. State: Replica: 1acdbab8d93242a497efedb13ca98044, State: Running, Role: LEADER
I20260810 18:42:08.294811 13601 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1acdbab8d93242a497efedb13ca98044 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1acdbab8d93242a497efedb13ca98044" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46603 } }
I20260810 18:42:08.295531 13594 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1acdbab8d93242a497efedb13ca98044 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:42:08.299314 13602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1acdbab8d93242a497efedb13ca98044 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1acdbab8d93242a497efedb13ca98044" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1acdbab8d93242a497efedb13ca98044" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46603 } } }
I20260810 18:42:08.299770 13602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1acdbab8d93242a497efedb13ca98044 [sys.catalog]: This master's current role is: LEADER
I20260810 18:42:08.299324 13603 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1acdbab8d93242a497efedb13ca98044 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1acdbab8d93242a497efedb13ca98044. Latest consensus state: current_term: 1 leader_uuid: "1acdbab8d93242a497efedb13ca98044" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1acdbab8d93242a497efedb13ca98044" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46603 } } }
I20260810 18:42:08.299970 13603 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1acdbab8d93242a497efedb13ca98044 [sys.catalog]: This master's current role is: LEADER
I20260810 18:42:08.301129 13613 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:42:08.307440 13613 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:42:08.316836 13613 catalog_manager.cc:1347] Generated new cluster ID: 68a6846ac80c46a39aa7f96df1d9722b
I20260810 18:42:08.316978 13613 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:42:08.327020 13613 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:42:08.329243 13613 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:42:08.342214 13613 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1acdbab8d93242a497efedb13ca98044: Generated new TSK 0
I20260810 18:42:08.343516 13613 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.228.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab.
Entry for principal kudu/127.4.228.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab.
I20260810 18:42:08.416363 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.193:0
--local_ip_for_outbound_sockets=127.4.228.193
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:46603
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:41467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:42:08.782814 13631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:42:08.783213 13631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:42:08.783392 13631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:42:08.791082 13631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:42:08.791214 13631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:42:08.791395 13631 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.193
I20260810 18:42:08.802380 13631 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:41467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:46603
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.193
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:42:08.804072 13631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:42:08.805873 13631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:42:08.820698 13645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:08.820617 13643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:08.820644 13641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:42:08.821568 13631 server_base.cc:1034] running on GCE node
I20260810 18:42:08.822252 13631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:42:08.823370 13631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:42:08.824651 13631 hybrid_clock.cc:648] HybridClock initialized: now 1786387328824547 us; error 88 us; skew 500 ppm
I20260810 18:42:08.825049 13631 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:42:08 dist-test-slave-7kzw krb5kdc[13471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387328, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:42:08.828637 13631 init.cc:377] Logged in from keytab as kudu/127.4.228.193@KRBTEST.COM (short username kudu)
I20260810 18:42:08.830330 13631 webserver.cc:460] Webserver started at http://127.4.228.193:34779/ using document root <none> and password file <none>
I20260810 18:42:08.831202 13631 fs_manager.cc:359] Metadata directory not provided
I20260810 18:42:08.831362 13631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:42:08.831717 13631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:42:08.834983 13631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-0/data/instance:
uuid: "85e4aab49c994b8ebaa5d17fadb11950"
format_stamp: "Formatted at 2026-08-10 18:42:08 on dist-test-slave-7kzw"
server_key: "b1352e25caf5e71d9a7e010367f68db5"
server_key_iv: "772f1374f08462482e298559dc4f710d"
server_key_version: "encryptionkey@0"
I20260810 18:42:08.835855 13631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance:
uuid: "85e4aab49c994b8ebaa5d17fadb11950"
format_stamp: "Formatted at 2026-08-10 18:42:08 on dist-test-slave-7kzw"
server_key: "b1352e25caf5e71d9a7e010367f68db5"
server_key_iv: "772f1374f08462482e298559dc4f710d"
server_key_version: "encryptionkey@0"
I20260810 18:42:08.841748 13631 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260810 18:42:08.845960 13657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:08.847837 13631 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260810 18:42:08.848083 13631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-0/wal
uuid: "85e4aab49c994b8ebaa5d17fadb11950"
format_stamp: "Formatted at 2026-08-10 18:42:08 on dist-test-slave-7kzw"
server_key: "b1352e25caf5e71d9a7e010367f68db5"
server_key_iv: "772f1374f08462482e298559dc4f710d"
server_key_version: "encryptionkey@0"
I20260810 18:42:08.848366 13631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:42:08.867713 13631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:42:08.872889 13631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:42:08.873268 13631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:42:08.874677 13631 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:42:08.876392 13631 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:42:08.876520 13631 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:08.876722 13631 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:42:08.876811 13631 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:08.915429 13631 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.193:39067
I20260810 18:42:08.915486 13817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.193:39067 every 8 connection(s)
I20260810 18:42:08.917062 13631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
I20260810 18:42:08.926431 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 13631
I20260810 18:42:08.926756 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance
I20260810 18:42:08.927716 5011 external_mini_cluster.cc:1428] Setting key 9b1f040fe0dfcd37b0542b294ddca79f
Aug 10 18:42:08 dist-test-slave-7kzw krb5kdc[13471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387328, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:42:08.943563 13822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:08.920109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:51183 (local address 127.4.228.254:46603)
0810 18:42:08.920669 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:08.920674 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:08.921912 (+ 1238us) server_negotiation.cc:407] Connection header received
0810 18:42:08.923997 (+ 2085us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:08.924002 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:08.924082 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:08.924224 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:08.926014 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:08.926974 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:08.929487 (+ 2513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:08.930222 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:08.930391 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:08.935981 (+ 5590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:08.935988 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:08.935992 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:08.936037 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:08.939269 (+ 3232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:08.940561 (+ 1292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:08.940566 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:08.940569 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:08.940714 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:08.941335 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:08.941340 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:08.941342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:08.941657 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:08.941793 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:08.943003 (+ 1210us) server_negotiation.cc:299] Negotiation successful
0810 18:42:08.943257 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":273,"threads_started":1}
I20260810 18:42:08.945350 13823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:08.920785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:46603 (local address 127.4.228.193:51183)
0810 18:42:08.921703 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:08.921758 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:08.923664 (+ 1906us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:08.924498 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:08.924507 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:08.925119 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:08.925781 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:08.925802 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:08.927184 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:08.927193 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:08.929303 (+ 2110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:08.929318 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:08.930575 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:08.930587 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:08.930951 (+ 364us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:08.931888 (+ 937us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:08.931929 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:08.935650 (+ 3721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:08.939530 (+ 3880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:08.939546 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:08.939572 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:08.940392 (+ 820us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:08.940847 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:08.940855 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:08.940862 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:08.941193 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:08.941825 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:08.941839 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:08.942441 (+ 602us) client_negotiation.cc:769] Sending connection context
0810 18:42:08.942893 (+ 452us) client_negotiation.cc:240] Negotiation successful
0810 18:42:08.943617 (+ 724us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":393,"threads_started":1}
I20260810 18:42:08.947420 13819 heartbeater.cc:347] Connected to a master server at 127.4.228.254:46603
I20260810 18:42:08.947803 13819 heartbeater.cc:464] Registering TS with master...
I20260810 18:42:08.948690 13819 heartbeater.cc:511] Master 127.4.228.254:46603 requested a full tablet report, sending...
I20260810 18:42:08.951196 13537 ts_manager.cc:194] Registered new tserver with Master: 85e4aab49c994b8ebaa5d17fadb11950 (127.4.228.193:39067)
I20260810 18:42:08.953060 13537 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.193@KRBTEST.COM'} at 127.4.228.193:51183
WARNING: no policy specified for kudu/127.4.228.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab.
Entry for principal kudu/127.4.228.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab.
I20260810 18:42:08.999025 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.194:0
--local_ip_for_outbound_sockets=127.4.228.194
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:46603
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:41467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:42:09.367631 13831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:42:09.368021 13831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:42:09.368198 13831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:42:09.375753 13831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:42:09.375914 13831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:42:09.376096 13831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.194
I20260810 18:42:09.388175 13831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:41467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:46603
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.194
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:42:09.389672 13831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:42:09.391218 13831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:42:09.405680 13843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:09.405680 13845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:09.405932 13847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:42:09.406394 13831 server_base.cc:1034] running on GCE node
I20260810 18:42:09.407125 13831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:42:09.408205 13831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:42:09.409451 13831 hybrid_clock.cc:648] HybridClock initialized: now 1786387329409386 us; error 63 us; skew 500 ppm
I20260810 18:42:09.409818 13831 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:42:09 dist-test-slave-7kzw krb5kdc[13471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387329, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:42:09.413185 13831 init.cc:377] Logged in from keytab as kudu/127.4.228.194@KRBTEST.COM (short username kudu)
I20260810 18:42:09.414925 13831 webserver.cc:460] Webserver started at http://127.4.228.194:34223/ using document root <none> and password file <none>
I20260810 18:42:09.415632 13831 fs_manager.cc:359] Metadata directory not provided
I20260810 18:42:09.415748 13831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:42:09.416107 13831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:42:09.419394 13831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-1/data/instance:
uuid: "152639b1f1604d6c82527d44346dff47"
format_stamp: "Formatted at 2026-08-10 18:42:09 on dist-test-slave-7kzw"
server_key: "da8f635ae677dc3993e791c2a4b1644e"
server_key_iv: "8eb11a103c71874bd89f18dd065c5352"
server_key_version: "encryptionkey@0"
I20260810 18:42:09.420199 13831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance:
uuid: "152639b1f1604d6c82527d44346dff47"
format_stamp: "Formatted at 2026-08-10 18:42:09 on dist-test-slave-7kzw"
server_key: "da8f635ae677dc3993e791c2a4b1644e"
server_key_iv: "8eb11a103c71874bd89f18dd065c5352"
server_key_version: "encryptionkey@0"
I20260810 18:42:09.425740 13831 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 18:42:09.429680 13859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:09.431202 13831 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 18:42:09.431360 13831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-1/wal
uuid: "152639b1f1604d6c82527d44346dff47"
format_stamp: "Formatted at 2026-08-10 18:42:09 on dist-test-slave-7kzw"
server_key: "da8f635ae677dc3993e791c2a4b1644e"
server_key_iv: "8eb11a103c71874bd89f18dd065c5352"
server_key_version: "encryptionkey@0"
I20260810 18:42:09.431566 13831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:42:09.445883 13831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:42:09.450852 13831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:42:09.451189 13831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:42:09.452337 13831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:42:09.454067 13831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:42:09.454160 13831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:09.454312 13831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:42:09.454373 13831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:09.493417 13831 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.194:36609
I20260810 18:42:09.493467 14026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.194:36609 every 8 connection(s)
I20260810 18:42:09.494985 13831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
I20260810 18:42:09.499308 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 13831
I20260810 18:42:09.499616 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance
I20260810 18:42:09.500167 5011 external_mini_cluster.cc:1428] Setting key f0a54970cc5df613b9cdbbe88e9b4e64
Aug 10 18:42:09 dist-test-slave-7kzw krb5kdc[13471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387329, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:42:09.520154 14031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:09.498333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:37863 (local address 127.4.228.254:46603)
0810 18:42:09.498861 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:09.498866 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:09.499999 (+ 1133us) server_negotiation.cc:407] Connection header received
0810 18:42:09.502350 (+ 2351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:09.502353 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:09.502458 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:09.502612 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:09.504355 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:09.505578 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:09.507508 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:09.508268 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:09.508435 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:09.512478 (+ 4043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:09.512487 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:09.512495 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:09.512547 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:09.515728 (+ 3181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:09.516955 (+ 1227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:09.516960 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:09.516962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:09.517129 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:09.517779 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:09.517783 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:09.517786 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:09.518201 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:09.518434 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:09.519550 (+ 1116us) server_negotiation.cc:299] Negotiation successful
0810 18:42:09.519812 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":306,"threads_started":1}
I20260810 18:42:09.521730 14033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:09.498838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:46603 (local address 127.4.228.194:37863)
0810 18:42:09.499808 (+ 970us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:09.499857 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:09.501991 (+ 2134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:09.502927 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:09.502944 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:09.503544 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:09.504166 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:09.504185 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:09.505820 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:09.505825 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:09.507337 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:09.507351 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:09.508540 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:09.508545 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:09.508801 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:09.509680 (+ 879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:09.509707 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:09.512205 (+ 2498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:09.515926 (+ 3721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:09.515940 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:09.515964 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:09.516737 (+ 773us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:09.517270 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:09.517278 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:09.517284 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:09.517572 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:09.518365 (+ 793us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:09.518378 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:09.519031 (+ 653us) client_negotiation.cc:769] Sending connection context
0810 18:42:09.519369 (+ 338us) client_negotiation.cc:240] Negotiation successful
0810 18:42:09.520044 (+ 675us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":393,"threads_started":1}
I20260810 18:42:09.523625 14027 heartbeater.cc:347] Connected to a master server at 127.4.228.254:46603
I20260810 18:42:09.524016 14027 heartbeater.cc:464] Registering TS with master...
I20260810 18:42:09.524991 14027 heartbeater.cc:511] Master 127.4.228.254:46603 requested a full tablet report, sending...
I20260810 18:42:09.527000 13537 ts_manager.cc:194] Registered new tserver with Master: 152639b1f1604d6c82527d44346dff47 (127.4.228.194:36609)
I20260810 18:42:09.528369 13537 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.194@KRBTEST.COM'} at 127.4.228.194:37863
WARNING: no policy specified for kudu/127.4.228.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab.
Entry for principal kudu/127.4.228.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab.
I20260810 18:42:09.567224 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.195:0
--local_ip_for_outbound_sockets=127.4.228.195
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:46603
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:41467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:42:09.942214 14037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:42:09.942638 14037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:42:09.942839 14037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:42:09.955808 14037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:42:09.955935 14037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:42:09.956127 14037 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.195
I20260810 18:42:09.957728 13819 heartbeater.cc:503] Master 127.4.228.254:46603 was elected leader, sending a full tablet report...
I20260810 18:42:09.967204 14037 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:41467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:46603
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.195
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:42:09.968735 14037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:42:09.970158 14037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:42:09.984560 14043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:09.984699 14044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:09.984627 14048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:42:09.986058 14037 server_base.cc:1034] running on GCE node
I20260810 18:42:09.986871 14037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:42:09.987978 14037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:42:09.989219 14037 hybrid_clock.cc:648] HybridClock initialized: now 1786387329989161 us; error 42 us; skew 500 ppm
I20260810 18:42:09.989607 14037 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:42:09 dist-test-slave-7kzw krb5kdc[13471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387329, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:42:09.993377 14037 init.cc:377] Logged in from keytab as kudu/127.4.228.195@KRBTEST.COM (short username kudu)
I20260810 18:42:09.995421 14037 webserver.cc:460] Webserver started at http://127.4.228.195:38089/ using document root <none> and password file <none>
I20260810 18:42:09.996213 14037 fs_manager.cc:359] Metadata directory not provided
I20260810 18:42:09.996395 14037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:42:09.996780 14037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:42:10.000231 14037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-2/data/instance:
uuid: "27c58285e08d4460be0f7c7dc3becdc1"
format_stamp: "Formatted at 2026-08-10 18:42:09 on dist-test-slave-7kzw"
server_key: "58182a542057017b0b10aa3a09d43ad0"
server_key_iv: "f5c3d038c0d2417166d69e0b04996714"
server_key_version: "encryptionkey@0"
I20260810 18:42:10.001083 14037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance:
uuid: "27c58285e08d4460be0f7c7dc3becdc1"
format_stamp: "Formatted at 2026-08-10 18:42:09 on dist-test-slave-7kzw"
server_key: "58182a542057017b0b10aa3a09d43ad0"
server_key_iv: "f5c3d038c0d2417166d69e0b04996714"
server_key_version: "encryptionkey@0"
I20260810 18:42:10.007151 14037 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260810 18:42:10.011543 14058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:10.013258 14037 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:42:10.013489 14037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-2/wal
uuid: "27c58285e08d4460be0f7c7dc3becdc1"
format_stamp: "Formatted at 2026-08-10 18:42:09 on dist-test-slave-7kzw"
server_key: "58182a542057017b0b10aa3a09d43ad0"
server_key_iv: "f5c3d038c0d2417166d69e0b04996714"
server_key_version: "encryptionkey@0"
I20260810 18:42:10.013737 14037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:42:10.039364 14037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:42:10.044258 14037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:42:10.044586 14037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:42:10.045706 14037 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:42:10.047365 14037 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:42:10.047478 14037 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:10.047638 14037 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:42:10.047716 14037 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:10.086910 14037 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.195:44007
I20260810 18:42:10.086975 14222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.195:44007 every 8 connection(s)
I20260810 18:42:10.088490 14037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
I20260810 18:42:10.098618 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 14037
I20260810 18:42:10.098896 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance
I20260810 18:42:10.099490 5011 external_mini_cluster.cc:1428] Setting key 7232007e0a7d2b51213a801023fe10fa
Aug 10 18:42:10 dist-test-slave-7kzw krb5kdc[13471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387329, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:42:10.114328 14226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:10.091750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:43403 (local address 127.4.228.254:46603)
0810 18:42:10.092332 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:10.092337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:10.093520 (+ 1183us) server_negotiation.cc:407] Connection header received
0810 18:42:10.095647 (+ 2127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:10.095655 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:10.095763 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:10.095894 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:10.097835 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:10.099328 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:10.101678 (+ 2350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:10.102285 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:10.102398 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:10.107125 (+ 4727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:10.107132 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:10.107136 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:10.107178 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:10.110344 (+ 3166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:10.111451 (+ 1107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:10.111456 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:10.111459 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:10.111663 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:10.112254 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:10.112259 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:10.112262 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:10.112626 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:10.112811 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:10.113565 (+ 754us) server_negotiation.cc:299] Negotiation successful
0810 18:42:10.113909 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":309,"threads_started":1}
I20260810 18:42:10.115489 14227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:10.092495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:46603 (local address 127.4.228.195:43403)
0810 18:42:10.093331 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:10.093373 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:10.095340 (+ 1967us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:10.096182 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:10.096198 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:10.096816 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:10.097436 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:10.097456 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:10.099579 (+ 2123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:10.099588 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:10.101455 (+ 1867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:10.101474 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:10.102652 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:10.102666 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:10.103031 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:10.103908 (+ 877us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:10.103936 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:10.106777 (+ 2841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:10.110611 (+ 3834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:10.110622 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:10.110640 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:10.111296 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:10.111829 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:10.111833 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:10.111836 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:10.112075 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:10.112780 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:10.112786 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:10.113189 (+ 403us) client_negotiation.cc:769] Sending connection context
0810 18:42:10.113449 (+ 260us) client_negotiation.cc:240] Negotiation successful
0810 18:42:10.113910 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":345,"threads_started":1}
I20260810 18:42:10.117660 14224 heartbeater.cc:347] Connected to a master server at 127.4.228.254:46603
I20260810 18:42:10.118204 14224 heartbeater.cc:464] Registering TS with master...
I20260810 18:42:10.119385 14224 heartbeater.cc:511] Master 127.4.228.254:46603 requested a full tablet report, sending...
I20260810 18:42:10.121310 13537 ts_manager.cc:194] Registered new tserver with Master: 27c58285e08d4460be0f7c7dc3becdc1 (127.4.228.195:44007)
I20260810 18:42:10.122216 13537 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.195@KRBTEST.COM'} at 127.4.228.195:43403
I20260810 18:42:10.130146 5011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/krb5kdc/test-admin.keytab.
Aug 10 18:42:10 dist-test-slave-7kzw krb5kdc[13471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:42:10.166188 5011 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260810 18:42:10.172487 14232 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:10.169256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38076 (local address 127.4.228.193:39067)
0810 18:42:10.170193 (+ 937us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:10.170201 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:10.170254 (+ 53us) server_negotiation.cc:407] Connection header received
0810 18:42:10.170346 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:10.170354 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:10.170667 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:10.170821 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:10.171972 (+ 1151us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38076: BlockingRecv error: recv got EOF from 127.0.0.1:38076 (error 108)
Metrics: {"server-negotiator.queue_time_us":733,"thread_start_us":568,"threads_started":1}
W20260810 18:42:10.172936 5011 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/krb5kdc/krb5cc' of type FILE
Aug 10 18:42:10 dist-test-slave-7kzw krb5kdc[13471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:42:10.174873 5011 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 18:42:10 dist-test-slave-7kzw krb5kdc[13471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:42:10.176723 5011 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 18:42:10 dist-test-slave-7kzw krb5kdc[13471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.193@KRBTEST.COM
I20260810 18:42:10.197335 14232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:10.179196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38084 (local address 127.4.228.193:39067)
0810 18:42:10.179446 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:10.179452 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:10.179701 (+ 249us) server_negotiation.cc:407] Connection header received
0810 18:42:10.179970 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:10.179979 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:10.180099 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:10.180239 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:10.181717 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:10.182997 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:10.184375 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:10.184981 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:10.185133 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:10.190138 (+ 5005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:10.190166 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:10.190192 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:10.190264 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:10.193748 (+ 3484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:10.194632 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:10.194641 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:10.194644 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:10.194790 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:10.195278 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:10.195283 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:10.195285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:10.195843 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:10.196153 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:10.196587 (+ 434us) server_negotiation.cc:299] Negotiation successful
0810 18:42:10.196993 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
W20260810 18:42:10.204484 14232 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:10.202315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38100 (local address 127.4.228.193:39067)
0810 18:42:10.202583 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:10.202590 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:10.202836 (+ 246us) server_negotiation.cc:407] Connection header received
0810 18:42:10.203020 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:10.203034 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:10.203123 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:10.203218 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:10.204235 (+ 1017us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38100: BlockingRecv error: recv got EOF from 127.0.0.1:38100 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
W20260810 18:42:10.204720 5011 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/krb5kdc/krb5cc' of type FILE
Aug 10 18:42:10 dist-test-slave-7kzw krb5kdc[13471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:42:10.206784 5011 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 18:42:10 dist-test-slave-7kzw krb5kdc[13471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:42:10.208716 5011 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 18:42:10 dist-test-slave-7kzw krb5kdc[13471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.193@KRBTEST.COM
I20260810 18:42:10.227357 14232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:10.210367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38106 (local address 127.4.228.193:39067)
0810 18:42:10.210718 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:10.210727 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:10.211366 (+ 639us) server_negotiation.cc:407] Connection header received
0810 18:42:10.211498 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:10.211503 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:10.211590 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:10.211684 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:10.213014 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:10.213743 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:10.215343 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:10.215777 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:10.215884 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:10.220719 (+ 4835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:10.220726 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:10.220731 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:10.220791 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:10.224009 (+ 3218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:10.224906 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:10.224911 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:10.224913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:10.225167 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:10.225659 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:10.225663 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:10.225666 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:10.225964 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:10.226208 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:10.226714 (+ 506us) server_negotiation.cc:299] Negotiation successful
0810 18:42:10.226937 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260810 18:42:10.233104 14232 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:10.230982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38110 (local address 127.4.228.193:39067)
0810 18:42:10.231278 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:10.231285 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:10.231521 (+ 236us) server_negotiation.cc:407] Connection header received
0810 18:42:10.231603 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:10.231606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:10.231718 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:10.231803 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:10.232869 (+ 1066us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38110: BlockingRecv error: recv got EOF from 127.0.0.1:38110 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
W20260810 18:42:10.233382 5011 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387267874959-5011-0/krb5kdc/krb5cc' of type FILE
Aug 10 18:42:10 dist-test-slave-7kzw krb5kdc[13471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:42:10.235338 5011 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 18:42:10 dist-test-slave-7kzw krb5kdc[13471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:42:10.237048 5011 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 18:42:10 dist-test-slave-7kzw krb5kdc[13471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.193@KRBTEST.COM
I20260810 18:42:10.255563 14232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:10.238757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38112 (local address 127.4.228.193:39067)
0810 18:42:10.239048 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:10.239056 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:10.239332 (+ 276us) server_negotiation.cc:407] Connection header received
0810 18:42:10.239488 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:10.239492 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:10.239591 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:10.239706 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:10.240878 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:10.241988 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:10.243477 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:10.244193 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:10.244347 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:10.248622 (+ 4275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:10.248632 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:10.248640 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:10.248697 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:10.251741 (+ 3044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:10.252846 (+ 1105us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:10.252851 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:10.252854 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:10.253035 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:10.253510 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:10.253515 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:10.253517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:10.253873 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:10.254194 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:10.254668 (+ 474us) server_negotiation.cc:299] Negotiation successful
0810 18:42:10.254967 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260810 18:42:10.257563 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 13631
I20260810 18:42:10.271821 14232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:10.258593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38126 (local address 127.4.228.193:39067)
0810 18:42:10.258824 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:10.258833 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:10.259325 (+ 492us) server_negotiation.cc:407] Connection header received
0810 18:42:10.259449 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:10.259453 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:10.259516 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:10.259599 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:10.260722 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:10.261446 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:10.263013 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:10.263438 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:10.263597 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:10.266036 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:10.266042 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:10.266045 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:10.266089 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:10.268896 (+ 2807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:10.269659 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:10.269664 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:10.269667 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:10.269851 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:10.270250 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:10.270255 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:10.270257 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:10.270675 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:10.270900 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:10.271216 (+ 316us) server_negotiation.cc:299] Negotiation successful
0810 18:42:10.271433 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260810 18:42:10.272532 13812 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:42:10.359887 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 13631
I20260810 18:42:10.388691 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 13831
Aug 10 18:42:10 dist-test-slave-7kzw krb5kdc[13471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.194@KRBTEST.COM
I20260810 18:42:10.407253 14250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:10.389684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37670 (local address 127.4.228.194:36609)
0810 18:42:10.390242 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:10.390249 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:10.390274 (+ 25us) server_negotiation.cc:407] Connection header received
0810 18:42:10.390337 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:10.390343 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:10.390752 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:10.390941 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:10.392361 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:10.393413 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:10.395171 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:10.395803 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:10.395968 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:10.400721 (+ 4753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:10.400735 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:10.400753 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:10.400795 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:10.404116 (+ 3321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:10.404920 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:10.404928 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:10.404930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:10.405095 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:10.405513 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:10.405517 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:10.405519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:10.405948 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:10.406238 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:10.406695 (+ 457us) server_negotiation.cc:299] Negotiation successful
0810 18:42:10.406934 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":279,"threads_started":1}
I20260810 18:42:10.408696 14019 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:42:10.493989 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 13831
I20260810 18:42:10.519966 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 14037
Aug 10 18:42:10 dist-test-slave-7kzw krb5kdc[13471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387330, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.195@KRBTEST.COM
I20260810 18:42:10.537089 14253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:10.521037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41988 (local address 127.4.228.195:44007)
0810 18:42:10.521703 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:10.521711 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:10.521745 (+ 34us) server_negotiation.cc:407] Connection header received
0810 18:42:10.521818 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:10.521825 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:10.522044 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:10.522189 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:10.523478 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:10.524557 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:10.525895 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:10.526573 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:10.526731 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:10.531075 (+ 4344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:10.531089 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:10.531106 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:10.531157 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:10.534001 (+ 2844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:10.534810 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:10.534819 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:10.534821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:10.534951 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:10.535488 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:10.535492 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:10.535494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:10.535925 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:10.536180 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:10.536503 (+ 323us) server_negotiation.cc:299] Negotiation successful
0810 18:42:10.536754 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":309,"threads_started":1}
I20260810 18:42:10.538523 14215 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:42:10.623373 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 14037
I20260810 18:42:10.651746 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 13489
I20260810 18:42:10.653384 13586 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:42:10.764418 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 13489
2026-08-10T18:42:10Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4728 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:42:10 dist-test-slave-7kzw krb5kdc[14258](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:42:10 dist-test-slave-7kzw krb5kdc[14258](info): set up 2 sockets
Aug 10 18:42:10 dist-test-slave-7kzw krb5kdc[14258](info): commencing operation
krb5kdc: starting...
W20260810 18:42:12.147893 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.328s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:42:12 dist-test-slave-7kzw krb5kdc[14258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:42:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:42:12Z Disabled control of system clock
WARNING: no policy specified for oryx/127.4.228.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.228.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.228.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.254.keytab.
Entry for principal oryx/127.4.228.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.254.keytab.
I20260810 18:42:12.452175 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:45361
--webserver_interface=127.4.228.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.254.keytab
--principal=oryx/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:36525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:45361
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:42:12.826983 14282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:42:12.827436 14282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:42:12.827622 14282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:42:12.835753 14282 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:42:12.835952 14282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:42:12.836073 14282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:42:12.836199 14282 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:42:12.836383 14282 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:42:12.847697 14282 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:36525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:45361
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.254.keytab
--principal=oryx/127.4.228.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:45361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:42:12.849344 14282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:42:12.850857 14282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:42:12.865247 14292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:12.865396 14289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:12.865367 14287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:42:12.866892 14282 server_base.cc:1034] running on GCE node
I20260810 18:42:12.868074 14282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:42:12.869875 14282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:42:12.871188 14282 hybrid_clock.cc:648] HybridClock initialized: now 1786387332871144 us; error 66 us; skew 500 ppm
I20260810 18:42:12.871656 14282 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:42:12 dist-test-slave-7kzw krb5kdc[14258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387332, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:42:12.875507 14282 init.cc:377] Logged in from keytab as oryx/127.4.228.254@KRBTEST.COM (short username oryx)
I20260810 18:42:12.877398 14282 webserver.cc:460] Webserver started at http://127.4.228.254:41459/ using document root <none> and password file <none>
I20260810 18:42:12.878249 14282 fs_manager.cc:359] Metadata directory not provided
I20260810 18:42:12.878422 14282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:42:12.878801 14282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:42:12.882368 14282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "c2d38638ade14167b99bd0ceb219791e"
format_stamp: "Formatted at 2026-08-10 18:42:12 on dist-test-slave-7kzw"
server_key: "e82ff191969d8bfc2ed7a963b48b2b38"
server_key_iv: "d666c2cb5f51ae509bd173bd1d47a33d"
server_key_version: "encryptionkey@0"
I20260810 18:42:12.883391 14282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "c2d38638ade14167b99bd0ceb219791e"
format_stamp: "Formatted at 2026-08-10 18:42:12 on dist-test-slave-7kzw"
server_key: "e82ff191969d8bfc2ed7a963b48b2b38"
server_key_iv: "d666c2cb5f51ae509bd173bd1d47a33d"
server_key_version: "encryptionkey@0"
I20260810 18:42:12.889775 14282 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260810 18:42:12.894194 14305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:12.896060 14282 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 18:42:12.896338 14282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "c2d38638ade14167b99bd0ceb219791e"
format_stamp: "Formatted at 2026-08-10 18:42:12 on dist-test-slave-7kzw"
server_key: "e82ff191969d8bfc2ed7a963b48b2b38"
server_key_iv: "d666c2cb5f51ae509bd173bd1d47a33d"
server_key_version: "encryptionkey@0"
I20260810 18:42:12.896772 14282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:42:12.920778 14282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:42:12.925822 14282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:42:12.926201 14282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:42:12.947805 14282 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:45361
I20260810 18:42:12.947811 14382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:45361 every 8 connection(s)
I20260810 18:42:12.949445 14282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:42:12.952751 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 14282
I20260810 18:42:12.953052 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:42:12.953860 5011 external_mini_cluster.cc:1428] Setting key c205dbbbbcb7a1d604fd83499ea10112
I20260810 18:42:12.954121 14384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:42:12.970665 14384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e: Bootstrap starting.
Aug 10 18:42:12 dist-test-slave-7kzw krb5kdc[14258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM
I20260810 18:42:12.975231 14384 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e: Neither blocks nor log segments found. Creating new log.
I20260810 18:42:12.976816 14384 log.cc:824] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e: Log is configured to *not* fsync() on all Append() calls
I20260810 18:42:12.981122 14384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e: No bootstrap required, opened a new log
I20260810 18:42:12.982615 14387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:12.956312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44888 (local address 127.4.228.254:45361)
0810 18:42:12.957281 (+ 969us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:12.957294 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:12.957345 (+ 51us) server_negotiation.cc:407] Connection header received
0810 18:42:12.959476 (+ 2131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:12.959504 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:12.960017 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:12.960420 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:12.962632 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:12.964019 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:12.965957 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:12.966869 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:12.967034 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:12.972303 (+ 5269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:12.972319 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:12.972350 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:12.972420 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:12.976998 (+ 4578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:12.978302 (+ 1304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:12.978311 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:12.978319 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:12.978608 (+ 289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:12.979046 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:12.979051 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:12.979054 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:12.979827 (+ 773us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:12.980178 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:12.980591 (+ 413us) server_negotiation.cc:299] Negotiation successful
0810 18:42:12.981071 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":718,"thread_start_us":439,"threads_started":1}
I20260810 18:42:12.990338 14384 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } }
I20260810 18:42:12.990854 14384 raft_consensus.cc:374] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:42:12.990975 14384 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2d38638ade14167b99bd0ceb219791e, State: Initialized, Role: FOLLOWER
I20260810 18:42:12.991729 14384 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } }
I20260810 18:42:12.992031 14384 raft_consensus.cc:388] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:42:12.992188 14384 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:42:12.992381 14384 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:42:12.994781 14384 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } }
I20260810 18:42:12.995390 14384 leader_election.cc:302] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c2d38638ade14167b99bd0ceb219791e; no voters:
I20260810 18:42:12.996196 14384 leader_election.cc:288] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:42:12.996354 14389 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:42:12.997315 14389 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [term 1 LEADER]: Becoming Leader. State: Replica: c2d38638ade14167b99bd0ceb219791e, State: Running, Role: LEADER
I20260810 18:42:12.997959 14389 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } }
I20260810 18:42:12.998787 14384 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:42:13.001785 14393 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2d38638ade14167b99bd0ceb219791e. Latest consensus state: current_term: 1 leader_uuid: "c2d38638ade14167b99bd0ceb219791e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } }
I20260810 18:42:13.001806 14391 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c2d38638ade14167b99bd0ceb219791e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } }
I20260810 18:42:13.002308 14393 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [sys.catalog]: This master's current role is: LEADER
I20260810 18:42:13.002347 14391 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [sys.catalog]: This master's current role is: LEADER
I20260810 18:42:13.004357 14401 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:42:13.010290 14401 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:42:13.020998 14401 catalog_manager.cc:1347] Generated new cluster ID: df523c3658514fd88ae4a2bfce9ed027
I20260810 18:42:13.021153 14401 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:42:13.034893 14401 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:42:13.037109 14401 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:42:13.050006 14401 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e: Generated new TSK 0
I20260810 18:42:13.051358 14401 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.4.228.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.228.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.228.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.193.keytab.
Entry for principal oryx/127.4.228.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.193.keytab.
I20260810 18:42:13.123759 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.193:0
--local_ip_for_outbound_sockets=127.4.228.193
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:45361
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.193.keytab
--principal=oryx/127.4.228.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:36525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:42:13.513120 14424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:42:13.513530 14424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:42:13.513717 14424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:42:13.521590 14424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:42:13.521811 14424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:42:13.522014 14424 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.193
I20260810 18:42:13.532686 14424 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:36525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.193.keytab
--principal=oryx/127.4.228.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:45361
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.193
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:42:13.534310 14424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:42:13.536109 14424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:42:13.553061 14431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:13.553092 14435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:13.553303 14432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:42:13.554932 14424 server_base.cc:1034] running on GCE node
I20260810 18:42:13.555773 14424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:42:13.556931 14424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:42:13.558192 14424 hybrid_clock.cc:648] HybridClock initialized: now 1786387333558131 us; error 49 us; skew 500 ppm
I20260810 18:42:13.558657 14424 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:42:13 dist-test-slave-7kzw krb5kdc[14258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387333, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:42:13.566224 14424 init.cc:377] Logged in from keytab as oryx/127.4.228.193@KRBTEST.COM (short username oryx)
I20260810 18:42:13.568437 14424 webserver.cc:460] Webserver started at http://127.4.228.193:41307/ using document root <none> and password file <none>
I20260810 18:42:13.569345 14424 fs_manager.cc:359] Metadata directory not provided
I20260810 18:42:13.569526 14424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:42:13.569892 14424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:42:13.573427 14424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-0/data/instance:
uuid: "6a9e3573e79244e681476d93147477ff"
format_stamp: "Formatted at 2026-08-10 18:42:13 on dist-test-slave-7kzw"
server_key: "30bf8fc09bfe0a9bf65f924618efd682"
server_key_iv: "21af14d65a8f388955ff873164a8ec82"
server_key_version: "encryptionkey@0"
I20260810 18:42:13.574345 14424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance:
uuid: "6a9e3573e79244e681476d93147477ff"
format_stamp: "Formatted at 2026-08-10 18:42:13 on dist-test-slave-7kzw"
server_key: "30bf8fc09bfe0a9bf65f924618efd682"
server_key_iv: "21af14d65a8f388955ff873164a8ec82"
server_key_version: "encryptionkey@0"
I20260810 18:42:13.580821 14424 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260810 18:42:13.584967 14442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:13.586746 14424 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 18:42:13.586966 14424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-0/wal
uuid: "6a9e3573e79244e681476d93147477ff"
format_stamp: "Formatted at 2026-08-10 18:42:13 on dist-test-slave-7kzw"
server_key: "30bf8fc09bfe0a9bf65f924618efd682"
server_key_iv: "21af14d65a8f388955ff873164a8ec82"
server_key_version: "encryptionkey@0"
I20260810 18:42:13.587222 14424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:42:13.613934 14424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:42:13.619366 14424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:42:13.619709 14424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:42:13.620847 14424 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:42:13.622331 14424 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:42:13.622445 14424 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:13.622629 14424 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:42:13.622716 14424 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:13.659952 14424 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.193:41347
I20260810 18:42:13.660121 14610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.193:41347 every 8 connection(s)
I20260810 18:42:13.661711 14424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
I20260810 18:42:13.666424 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 14424
I20260810 18:42:13.666757 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance
I20260810 18:42:13.667773 5011 external_mini_cluster.cc:1428] Setting key 1a95a5eab1d420b1dc75b86c32c5fca8
Aug 10 18:42:13 dist-test-slave-7kzw krb5kdc[14258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387333, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM
I20260810 18:42:13.689810 14616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:13.665580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:50867 (local address 127.4.228.254:45361)
0810 18:42:13.666298 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:13.666305 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:13.667711 (+ 1406us) server_negotiation.cc:407] Connection header received
0810 18:42:13.670461 (+ 2750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:13.670514 (+ 53us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:13.670635 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:13.670822 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:13.673498 (+ 2676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:13.674946 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:13.677851 (+ 2905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:13.678540 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:13.678767 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:13.682973 (+ 4206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:13.682979 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:13.682983 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:13.683021 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:13.685823 (+ 2802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:13.686785 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:13.686791 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:13.686795 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:13.686935 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:13.687501 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:13.687510 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:13.687516 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:13.687959 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:13.688118 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:13.689006 (+ 888us) server_negotiation.cc:299] Negotiation successful
0810 18:42:13.689305 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":3020,"threads_started":1}
I20260810 18:42:13.690912 14617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:13.666332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.4.228.193:50867)
0810 18:42:13.667461 (+ 1129us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:13.667530 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:13.670055 (+ 2525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:13.671194 (+ 1139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:13.671212 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:13.672177 (+ 965us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:13.673225 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:13.673263 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:13.675227 (+ 1964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:13.675235 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:13.677576 (+ 2341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:13.677594 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:13.678816 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:13.678825 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:13.679136 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:13.679949 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:13.679975 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:13.682696 (+ 2721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:13.686008 (+ 3312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:13.686019 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:13.686048 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:13.686611 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:13.687067 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:13.687072 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:13.687077 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:13.687306 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:13.688151 (+ 845us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:13.688158 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:13.688542 (+ 384us) client_negotiation.cc:769] Sending connection context
0810 18:42:13.688815 (+ 273us) client_negotiation.cc:240] Negotiation successful
0810 18:42:13.689345 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"thread_start_us":396,"threads_started":1}
I20260810 18:42:13.692898 14612 heartbeater.cc:347] Connected to a master server at 127.4.228.254:45361
I20260810 18:42:13.693285 14612 heartbeater.cc:464] Registering TS with master...
I20260810 18:42:13.694139 14612 heartbeater.cc:511] Master 127.4.228.254:45361 requested a full tablet report, sending...
I20260810 18:42:13.696776 14335 ts_manager.cc:194] Registered new tserver with Master: 6a9e3573e79244e681476d93147477ff (127.4.228.193:41347)
WARNING: no policy specified for oryx/127.4.228.194@KRBTEST.COM; defaulting to no policy
I20260810 18:42:13.698541 14335 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.228.193@KRBTEST.COM'} at 127.4.228.193:50867
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.228.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.228.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.194.keytab.
Entry for principal oryx/127.4.228.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.194.keytab.
I20260810 18:42:13.735069 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.194:0
--local_ip_for_outbound_sockets=127.4.228.194
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:45361
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.194.keytab
--principal=oryx/127.4.228.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:36525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:42:14.111476 14621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:42:14.111883 14621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:42:14.112064 14621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:42:14.119777 14621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:42:14.119959 14621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:42:14.120159 14621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.194
I20260810 18:42:14.130685 14621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:36525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.194.keytab
--principal=oryx/127.4.228.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:45361
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.194
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:42:14.132818 14621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:42:14.134788 14621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:42:14.151079 14628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:14.151149 14630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:14.151144 14626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:42:14.152436 14621 server_base.cc:1034] running on GCE node
I20260810 18:42:14.153223 14621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:42:14.154418 14621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:42:14.155761 14621 hybrid_clock.cc:648] HybridClock initialized: now 1786387334155709 us; error 50 us; skew 500 ppm
I20260810 18:42:14.156201 14621 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:42:14 dist-test-slave-7kzw krb5kdc[14258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387334, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:42:14.159890 14621 init.cc:377] Logged in from keytab as oryx/127.4.228.194@KRBTEST.COM (short username oryx)
I20260810 18:42:14.161724 14621 webserver.cc:460] Webserver started at http://127.4.228.194:43247/ using document root <none> and password file <none>
I20260810 18:42:14.162624 14621 fs_manager.cc:359] Metadata directory not provided
I20260810 18:42:14.162784 14621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:42:14.163141 14621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:42:14.166610 14621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-1/data/instance:
uuid: "70f6632aa17f433ab1c0bf5502ea03a1"
format_stamp: "Formatted at 2026-08-10 18:42:14 on dist-test-slave-7kzw"
server_key: "888ebb509af3963bab608fa001601a02"
server_key_iv: "38fdf4b3749a0f128395065ae2b04a68"
server_key_version: "encryptionkey@0"
I20260810 18:42:14.167486 14621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance:
uuid: "70f6632aa17f433ab1c0bf5502ea03a1"
format_stamp: "Formatted at 2026-08-10 18:42:14 on dist-test-slave-7kzw"
server_key: "888ebb509af3963bab608fa001601a02"
server_key_iv: "38fdf4b3749a0f128395065ae2b04a68"
server_key_version: "encryptionkey@0"
I20260810 18:42:14.173862 14621 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260810 18:42:14.178824 14638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:14.180601 14621 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 18:42:14.180867 14621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-1/wal
uuid: "70f6632aa17f433ab1c0bf5502ea03a1"
format_stamp: "Formatted at 2026-08-10 18:42:14 on dist-test-slave-7kzw"
server_key: "888ebb509af3963bab608fa001601a02"
server_key_iv: "38fdf4b3749a0f128395065ae2b04a68"
server_key_version: "encryptionkey@0"
I20260810 18:42:14.181123 14621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:42:14.209298 14621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:42:14.214807 14621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:42:14.215164 14621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:42:14.216329 14621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:42:14.217962 14621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:42:14.218084 14621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:14.218266 14621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:42:14.218458 14621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:14.256488 14621 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.194:44749
I20260810 18:42:14.256543 14823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.194:44749 every 8 connection(s)
I20260810 18:42:14.257920 14621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
I20260810 18:42:14.266072 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 14621
I20260810 18:42:14.266319 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance
I20260810 18:42:14.266897 5011 external_mini_cluster.cc:1428] Setting key a2a4917ab0d9bc11814aa58a2b4a3028
Aug 10 18:42:14 dist-test-slave-7kzw krb5kdc[14258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387334, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM
I20260810 18:42:14.281798 14826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:14.261108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:41015 (local address 127.4.228.254:45361)
0810 18:42:14.261687 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:14.261692 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:14.262934 (+ 1242us) server_negotiation.cc:407] Connection header received
0810 18:42:14.265068 (+ 2134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:14.265074 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:14.265200 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:14.265338 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:14.267392 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:14.268366 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:14.270223 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:14.270783 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:14.270933 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:14.275092 (+ 4159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:14.275098 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:14.275102 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:14.275153 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:14.278321 (+ 3168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:14.279238 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:14.279246 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:14.279251 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:14.279400 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:14.279899 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:14.279907 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:14.279915 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:14.280202 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:14.280368 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:14.281133 (+ 765us) server_negotiation.cc:299] Negotiation successful
0810 18:42:14.281386 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":336,"threads_started":1}
I20260810 18:42:14.283113 14831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:14.261742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.4.228.194:41015)
0810 18:42:14.262747 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:14.262797 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:14.264731 (+ 1934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:14.265629 (+ 898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:14.265646 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:14.266320 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:14.267166 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:14.267188 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:14.268548 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:14.268554 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:14.270044 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:14.270059 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:14.271069 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:14.271077 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:14.271345 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:14.272172 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:14.272199 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:14.274797 (+ 2598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:14.278539 (+ 3742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:14.278551 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:14.278574 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:14.279085 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:14.279526 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:14.279531 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:14.279535 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:14.279767 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:14.280349 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:14.280358 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:14.280743 (+ 385us) client_negotiation.cc:769] Sending connection context
0810 18:42:14.281033 (+ 290us) client_negotiation.cc:240] Negotiation successful
0810 18:42:14.281650 (+ 617us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":339,"threads_started":1}
I20260810 18:42:14.285188 14824 heartbeater.cc:347] Connected to a master server at 127.4.228.254:45361
I20260810 18:42:14.285593 14824 heartbeater.cc:464] Registering TS with master...
I20260810 18:42:14.286777 14824 heartbeater.cc:511] Master 127.4.228.254:45361 requested a full tablet report, sending...
I20260810 18:42:14.288882 14335 ts_manager.cc:194] Registered new tserver with Master: 70f6632aa17f433ab1c0bf5502ea03a1 (127.4.228.194:44749)
I20260810 18:42:14.290349 14335 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.228.194@KRBTEST.COM'} at 127.4.228.194:41015
WARNING: no policy specified for oryx/127.4.228.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.228.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.228.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.195.keytab.
Entry for principal oryx/127.4.228.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.195.keytab.
I20260810 18:42:14.335893 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.195:0
--local_ip_for_outbound_sockets=127.4.228.195
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:45361
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.195.keytab
--principal=oryx/127.4.228.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:36525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:42:14.702693 14612 heartbeater.cc:503] Master 127.4.228.254:45361 was elected leader, sending a full tablet report...
W20260810 18:42:14.713017 14835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:42:14.713465 14835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:42:14.713675 14835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:42:14.721887 14835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:42:14.722082 14835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:42:14.722270 14835 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.195
I20260810 18:42:14.732743 14835 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:36525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.195.keytab
--principal=oryx/127.4.228.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:45361
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.195
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:42:14.734373 14835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:42:14.735967 14835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:42:14.750589 14846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:14.750625 14843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:14.750589 14842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:42:14.752429 14835 server_base.cc:1034] running on GCE node
I20260810 18:42:14.753294 14835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:42:14.754386 14835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:42:14.755699 14835 hybrid_clock.cc:648] HybridClock initialized: now 1786387334755650 us; error 44 us; skew 500 ppm
I20260810 18:42:14.756103 14835 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:42:14 dist-test-slave-7kzw krb5kdc[14258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387334, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:42:14.760115 14835 init.cc:377] Logged in from keytab as oryx/127.4.228.195@KRBTEST.COM (short username oryx)
I20260810 18:42:14.761834 14835 webserver.cc:460] Webserver started at http://127.4.228.195:34799/ using document root <none> and password file <none>
I20260810 18:42:14.762723 14835 fs_manager.cc:359] Metadata directory not provided
I20260810 18:42:14.762913 14835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:42:14.763263 14835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:42:14.766606 14835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-2/data/instance:
uuid: "b5068707091b42a5b99875e6389e158e"
format_stamp: "Formatted at 2026-08-10 18:42:14 on dist-test-slave-7kzw"
server_key: "a81f90666e805354117093ff9438679c"
server_key_iv: "7e85402bac5cb2526928474d981463fe"
server_key_version: "encryptionkey@0"
I20260810 18:42:14.767500 14835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance:
uuid: "b5068707091b42a5b99875e6389e158e"
format_stamp: "Formatted at 2026-08-10 18:42:14 on dist-test-slave-7kzw"
server_key: "a81f90666e805354117093ff9438679c"
server_key_iv: "7e85402bac5cb2526928474d981463fe"
server_key_version: "encryptionkey@0"
I20260810 18:42:14.773448 14835 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 18:42:14.777653 14857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:14.779301 14835 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:42:14.779513 14835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-2/wal
uuid: "b5068707091b42a5b99875e6389e158e"
format_stamp: "Formatted at 2026-08-10 18:42:14 on dist-test-slave-7kzw"
server_key: "a81f90666e805354117093ff9438679c"
server_key_iv: "7e85402bac5cb2526928474d981463fe"
server_key_version: "encryptionkey@0"
I20260810 18:42:14.779758 14835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:42:14.807960 14835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:42:14.812916 14835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:42:14.813261 14835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:42:14.814392 14835 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:42:14.815943 14835 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:42:14.816059 14835 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:14.816224 14835 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:42:14.816305 14835 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:14.855059 14835 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.195:40897
I20260810 18:42:14.855110 15025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.195:40897 every 8 connection(s)
I20260810 18:42:14.856593 14835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
I20260810 18:42:14.866549 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 14835
I20260810 18:42:14.866813 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance
I20260810 18:42:14.867326 5011 external_mini_cluster.cc:1428] Setting key 8235ba4c44aa797e3b5ab9d5be124db6
Aug 10 18:42:14 dist-test-slave-7kzw krb5kdc[14258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387334, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.195@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM
I20260810 18:42:14.881098 15028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:14.859651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:60053 (local address 127.4.228.254:45361)
0810 18:42:14.860194 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:14.860201 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:14.861414 (+ 1213us) server_negotiation.cc:407] Connection header received
0810 18:42:14.863472 (+ 2058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:14.863477 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:14.863573 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:14.863681 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:14.865558 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:14.866713 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:14.868792 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:14.869307 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:14.869462 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:14.873690 (+ 4228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:14.873696 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:14.873700 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:14.873742 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:14.877071 (+ 3329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:14.878096 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:14.878102 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:14.878105 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:14.878261 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:14.878918 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:14.878923 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:14.878926 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:14.879268 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:14.879427 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:14.880490 (+ 1063us) server_negotiation.cc:299] Negotiation successful
0810 18:42:14.880718 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":231,"threads_started":1}
I20260810 18:42:14.882277 15029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:14.860362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.4.228.195:60053)
0810 18:42:14.861221 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:14.861273 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:14.863144 (+ 1871us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:14.864034 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:14.864044 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:14.864661 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:14.865338 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:14.865359 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:14.866929 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:14.866933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:14.868568 (+ 1635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:14.868582 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:14.869629 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:14.869638 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:14.869964 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:14.870819 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:14.870844 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:14.873411 (+ 2567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:14.877327 (+ 3916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:14.877338 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:14.877357 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:14.877936 (+ 579us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:14.878441 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:14.878446 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:14.878448 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:14.878784 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:14.879482 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:14.879489 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:14.879920 (+ 431us) client_negotiation.cc:769] Sending connection context
0810 18:42:14.880293 (+ 373us) client_negotiation.cc:240] Negotiation successful
0810 18:42:14.880775 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":342,"threads_started":1}
I20260810 18:42:14.884092 15026 heartbeater.cc:347] Connected to a master server at 127.4.228.254:45361
I20260810 18:42:14.884452 15026 heartbeater.cc:464] Registering TS with master...
I20260810 18:42:14.885382 15026 heartbeater.cc:511] Master 127.4.228.254:45361 requested a full tablet report, sending...
I20260810 18:42:14.887184 14334 ts_manager.cc:194] Registered new tserver with Master: b5068707091b42a5b99875e6389e158e (127.4.228.195:40897)
I20260810 18:42:14.888298 14334 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.228.195@KRBTEST.COM'} at 127.4.228.195:60053
I20260810 18:42:14.897418 5011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.4.228.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.228.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.228.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.253.keytab.
Entry for principal oryx/127.4.228.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.253.keytab.
I20260810 18:42:14.970283 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.253:39135
--webserver_interface=127.4.228.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.253.keytab
--principal=oryx/127.4.228.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:36525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:45361,127.4.228.253:39135
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:42:15.294718 14824 heartbeater.cc:503] Master 127.4.228.254:45361 was elected leader, sending a full tablet report...
W20260810 18:42:15.343688 15034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:42:15.344091 15034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:42:15.344277 15034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:42:15.352322 15034 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:42:15.352480 15034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:42:15.352600 15034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:42:15.352707 15034 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:42:15.352793 15034 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:42:15.363353 15034 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:36525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:45361,127.4.228.253:39135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.253.keytab
--principal=oryx/127.4.228.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.253:39135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.228.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
W20260810 18:42:15.363794 15034 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.4.228.254:45361,127.4.228.253:39135'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260810 18:42:15.364967 15034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:42:15.366405 15034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:42:15.379392 15043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:15.379339 15048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:15.379437 15044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:42:15.380462 15034 server_base.cc:1034] running on GCE node
I20260810 18:42:15.381330 15034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:42:15.383075 15034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:42:15.384301 15034 hybrid_clock.cc:648] HybridClock initialized: now 1786387335384210 us; error 62 us; skew 500 ppm
I20260810 18:42:15.384809 15034 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:42:15 dist-test-slave-7kzw krb5kdc[14258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387335, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:42:15.388406 15034 init.cc:377] Logged in from keytab as oryx/127.4.228.253@KRBTEST.COM (short username oryx)
I20260810 18:42:15.390079 15034 webserver.cc:460] Webserver started at http://127.4.228.253:38025/ using document root <none> and password file <none>
I20260810 18:42:15.390885 15034 fs_manager.cc:359] Metadata directory not provided
I20260810 18:42:15.390982 15034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:42:15.391273 15034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:42:15.394680 15034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-1/data/instance:
uuid: "882afacf744b4b6b89526c8f0108387e"
format_stamp: "Formatted at 2026-08-10 18:42:15 on dist-test-slave-7kzw"
server_key: "5877beb787714755e801041ad5eb01e9"
server_key_iv: "1f69e9221f56d42ad7fcf0f9f63f34db"
server_key_version: "encryptionkey@0"
I20260810 18:42:15.395466 15034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-1/wal/instance:
uuid: "882afacf744b4b6b89526c8f0108387e"
format_stamp: "Formatted at 2026-08-10 18:42:15 on dist-test-slave-7kzw"
server_key: "5877beb787714755e801041ad5eb01e9"
server_key_iv: "1f69e9221f56d42ad7fcf0f9f63f34db"
server_key_version: "encryptionkey@0"
I20260810 18:42:15.401212 15034 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 18:42:15.405419 15056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:15.407552 15034 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260810 18:42:15.407743 15034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-1/wal
uuid: "882afacf744b4b6b89526c8f0108387e"
format_stamp: "Formatted at 2026-08-10 18:42:15 on dist-test-slave-7kzw"
server_key: "5877beb787714755e801041ad5eb01e9"
server_key_iv: "1f69e9221f56d42ad7fcf0f9f63f34db"
server_key_version: "encryptionkey@0"
I20260810 18:42:15.408013 15034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:42:15.424666 15034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:42:15.429415 15034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:42:15.429721 15034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:42:15.451400 15034 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.253:39135
I20260810 18:42:15.451445 15139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.253:39135 every 8 connection(s)
I20260810 18:42:15.452936 15034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-1/data/info.pb
I20260810 18:42:15.457283 15140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:42:15.461426 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 15034
I20260810 18:42:15.466701 15140 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:42:15 dist-test-slave-7kzw krb5kdc[14258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.228.253@KRBTEST.COM
Aug 10 18:42:15 dist-test-slave-7kzw krb5kdc[14258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387335, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.253@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM
I20260810 18:42:15.488415 15148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:15.470076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38760 (local address 127.4.228.254:45361)
0810 18:42:15.470758 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:15.470764 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:15.471356 (+ 592us) server_negotiation.cc:407] Connection header received
0810 18:42:15.471569 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:15.471574 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:15.471711 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:15.471803 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:15.473199 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:15.474014 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:15.475688 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:15.476121 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:15.476275 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:15.480761 (+ 4486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:15.480774 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:15.480781 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:15.480836 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:15.484653 (+ 3817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:15.485575 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:15.485584 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:15.485588 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:15.485758 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:15.486226 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:15.486234 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:15.486241 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:15.486748 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:15.486934 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:15.487605 (+ 671us) server_negotiation.cc:299] Negotiation successful
0810 18:42:15.488000 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":314,"threads_started":1}
I20260810 18:42:15.489076 15147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:15.464379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38866 (local address 127.4.228.253:39135)
0810 18:42:15.465230 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:15.465243 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:15.465288 (+ 45us) server_negotiation.cc:407] Connection header received
0810 18:42:15.467184 (+ 1896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:15.467210 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:15.467672 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:15.468067 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:15.469777 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:15.472188 (+ 2411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:15.473555 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:15.474238 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:15.474470 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:15.479400 (+ 4930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:15.479413 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:15.479435 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:15.479484 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:15.483642 (+ 4158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:15.484511 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:15.484522 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:15.484534 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:15.484782 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:15.485304 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:15.485313 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:15.485316 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:15.486189 (+ 873us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:15.486557 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:15.487186 (+ 629us) server_negotiation.cc:299] Negotiation successful
0810 18:42:15.487673 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"spinlock_wait_cycles":18176,"thread_start_us":377,"threads_started":1}
I20260810 18:42:15.489125 15152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:15.470067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.0.0.1:38760)
0810 18:42:15.471175 (+ 1108us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:15.471235 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:15.471422 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:15.471907 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:15.471914 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:15.472757 (+ 843us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:15.473021 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:15.473031 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:15.474205 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:15.474209 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:15.475503 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:15.475517 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:15.476396 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:15.476407 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:15.476589 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:15.477335 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:15.477361 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:15.480519 (+ 3158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:15.484831 (+ 4312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:15.484846 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:15.484856 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:15.485426 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:15.485905 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:15.485910 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:15.485914 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:15.486054 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:15.486906 (+ 852us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:15.486915 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:15.487159 (+ 244us) client_negotiation.cc:769] Sending connection context
0810 18:42:15.487436 (+ 277us) client_negotiation.cc:240] Negotiation successful
0810 18:42:15.487873 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":817,"thread_start_us":622,"threads_started":1}
I20260810 18:42:15.491317 15140 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:42:15 dist-test-slave-7kzw krb5kdc[14258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387335, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.253@KRBTEST.COM for oryx/127.4.228.253@KRBTEST.COM
I20260810 18:42:15.506109 15152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:15.492306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:39135 (local address 127.0.0.1:38882)
0810 18:42:15.492501 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:15.492522 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:15.492641 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:15.493187 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:15.493190 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:15.493608 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:15.493820 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:15.493834 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:15.494995 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:15.495000 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:15.496034 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:15.496047 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:15.496942 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:15.496946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:15.497091 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:15.497781 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:15.497803 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:15.500415 (+ 2612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:15.503613 (+ 3198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:15.503619 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:15.503622 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:15.504079 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:15.504441 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:15.504444 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:15.504447 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:15.504590 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:15.505126 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:15.505129 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:15.505275 (+ 146us) client_negotiation.cc:769] Sending connection context
0810 18:42:15.505491 (+ 216us) client_negotiation.cc:240] Negotiation successful
0810 18:42:15.505789 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 18:42:15.506199 15147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:15.492810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38882 (local address 127.4.228.253:39135)
0810 18:42:15.492953 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:15.492957 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:15.492971 (+ 14us) server_negotiation.cc:407] Connection header received
0810 18:42:15.493006 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:15.493009 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:15.493063 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:15.493168 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:15.493990 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:15.494778 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:15.496217 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:15.496741 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:15.496879 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:15.500626 (+ 3747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:15.500631 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:15.500635 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:15.500667 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:15.503449 (+ 2782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:15.504200 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:15.504204 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:15.504206 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:15.504335 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:15.504719 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:15.504723 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:15.504725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:15.505011 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:15.505131 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:15.505682 (+ 551us) server_negotiation.cc:299] Negotiation successful
0810 18:42:15.505905 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":57}
I20260810 18:42:15.511803 15140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e: Bootstrap starting.
I20260810 18:42:15.515252 15140 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e: Neither blocks nor log segments found. Creating new log.
I20260810 18:42:15.516363 15140 log.cc:824] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e: Log is configured to *not* fsync() on all Append() calls
I20260810 18:42:15.519368 15140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e: No bootstrap required, opened a new log
I20260810 18:42:15.526755 15140 raft_consensus.cc:348] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } }
I20260810 18:42:15.527158 15140 raft_consensus.cc:374] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:42:15.527271 15140 raft_consensus.cc:729] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 882afacf744b4b6b89526c8f0108387e, State: Initialized, Role: FOLLOWER
I20260810 18:42:15.527906 15140 consensus_queue.cc:260] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } }
I20260810 18:42:15.528838 15158 sys_catalog.cc:455] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } } }
I20260810 18:42:15.529323 15158 sys_catalog.cc:458] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:42:15.529922 15140 sys_catalog.cc:564] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [sys.catalog]: configured and running, proceeding with master startup.
W20260810 18:42:15.541679 15172 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:42:15.541838 15172 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:42:15.545807 15034 master_runner.cc:428] Detected that this master 882afacf744b4b6b89526c8f0108387e is joining an existing cluster
I20260810 18:42:15.545957 15034 master_runner.cc:432] Initiating AddMaster RPC to add 127.4.228.253:39135
I20260810 18:42:15.568570 15148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:15.551326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38774 (local address 127.4.228.254:45361)
0810 18:42:15.551540 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:15.551549 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:15.552032 (+ 483us) server_negotiation.cc:407] Connection header received
0810 18:42:15.552328 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:15.552334 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:15.552422 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:15.552496 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:15.554096 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:15.555219 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:15.557383 (+ 2164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:15.558083 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:15.558203 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:15.561126 (+ 2923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:15.561134 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:15.561139 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:15.561183 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:15.565039 (+ 3856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:15.565958 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:15.565964 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:15.565968 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:15.566148 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:15.566669 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:15.566675 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:15.566678 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:15.567072 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:15.567206 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:15.567903 (+ 697us) server_negotiation.cc:299] Negotiation successful
0810 18:42:15.568185 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 18:42:15.570241 15181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:15.550935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.0.0.1:38774)
0810 18:42:15.551926 (+ 991us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:15.551947 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:15.552164 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:15.552736 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:15.552741 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:15.553455 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:15.553889 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:15.553904 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:15.555411 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:15.555416 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:15.557167 (+ 1751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:15.557183 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:15.558548 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:15.558556 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:15.558767 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:15.559550 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:15.559576 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:15.560880 (+ 1304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:15.565239 (+ 4359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:15.565249 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:15.565253 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:15.565803 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:15.566311 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:15.566316 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:15.566320 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:15.566530 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:15.567334 (+ 804us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:15.567338 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:15.567582 (+ 244us) client_negotiation.cc:769] Sending connection context
0810 18:42:15.569173 (+ 1591us) client_negotiation.cc:240] Negotiation successful
0810 18:42:15.569713 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":838,"thread_start_us":667,"threads_started":1}
I20260810 18:42:15.580339 15147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:15.550942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38894 (local address 127.4.228.253:39135)
0810 18:42:15.551192 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:15.551202 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:15.560563 (+ 9361us) server_negotiation.cc:407] Connection header received
0810 18:42:15.560792 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:15.560800 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:15.560891 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:15.560996 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:15.562623 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:15.564051 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:15.566730 (+ 2679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:15.567682 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:15.567808 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:15.570897 (+ 3089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:15.570908 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:15.570917 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:15.570968 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:15.576374 (+ 5406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:15.577437 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:15.577444 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:15.577450 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:15.577648 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:15.578174 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:15.578180 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:15.578184 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:15.578714 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:15.578895 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:15.579523 (+ 628us) server_negotiation.cc:299] Negotiation successful
0810 18:42:15.579857 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 18:42:15.581676 15182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:15.551380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:39135 (local address 127.0.0.1:38894)
0810 18:42:15.560304 (+ 8924us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:15.560322 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:15.560645 (+ 323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:15.561185 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:15.561192 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:15.561987 (+ 795us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:15.562404 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:15.562425 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:15.564322 (+ 1897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:15.564328 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:15.566520 (+ 2192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:15.566543 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:15.568169 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:15.568180 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:15.568425 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:15.569110 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:15.569145 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:15.570680 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:15.576542 (+ 5862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:15.576551 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:15.576558 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:15.577289 (+ 731us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:15.577835 (+ 546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:15.577843 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:15.577849 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:15.578039 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:15.578985 (+ 946us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:15.578992 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:15.579188 (+ 196us) client_negotiation.cc:769] Sending connection context
0810 18:42:15.580753 (+ 1565us) client_negotiation.cc:240] Negotiation successful
0810 18:42:15.581088 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8747,"thread_start_us":8519,"threads_started":1}
WARNING: no policy specified for oryx/127.4.228.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.228.252@KRBTEST.COM" created.
Aug 10 18:42:15 dist-test-slave-7kzw krb5kdc[14258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387332, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.254@KRBTEST.COM for oryx/127.4.228.253@KRBTEST.COM
I20260810 18:42:15.615741 15188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:15.578947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:39135 (local address 127.0.0.1:38900)
0810 18:42:15.592804 (+ 13857us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:15.592865 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:15.593332 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:15.594522 (+ 1190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:15.594535 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:15.595759 (+ 1224us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:15.596267 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:15.596287 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:15.598036 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:15.598044 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:15.600226 (+ 2182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:15.600258 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:15.601785 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:15.601793 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:15.602020 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:15.603084 (+ 1064us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:15.603111 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:15.606774 (+ 3663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:15.611970 (+ 5196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:15.611989 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:15.611999 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:15.612814 (+ 815us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:15.613337 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:15.613343 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:15.613348 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:15.613536 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:15.614383 (+ 847us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:15.614394 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:15.614644 (+ 250us) client_negotiation.cc:769] Sending connection context
0810 18:42:15.614874 (+ 230us) client_negotiation.cc:240] Negotiation successful
0810 18:42:15.615258 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13491,"thread_start_us":13319,"threads_started":1}
I20260810 18:42:15.615844 15147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:15.579372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38900 (local address 127.4.228.253:39135)
0810 18:42:15.580710 (+ 1338us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:15.580717 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:15.593081 (+ 12364us) server_negotiation.cc:407] Connection header received
0810 18:42:15.593701 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:15.593711 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:15.593862 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:15.594084 (+ 222us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:15.596503 (+ 2419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:15.597809 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:15.600475 (+ 2666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:15.601470 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:15.601647 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:15.607052 (+ 5405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:15.607062 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:15.607072 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:15.607132 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:15.611767 (+ 4635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:15.612987 (+ 1220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:15.612996 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:15.613003 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:15.613199 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:15.613689 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:15.613696 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:15.613702 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:15.614203 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:15.614372 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:15.615040 (+ 668us) server_negotiation.cc:299] Negotiation successful
0810 18:42:15.615329 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1196,"thread_start_us":12558,"threads_started":1}
I20260810 18:42:15.618938 14334 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.4.228.253:39135: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "882afacf744b4b6b89526c8f0108387e"
member_type: NON_VOTER
last_known_addr {
host: "127.4.228.253"
port: 39135
}
attrs {
promote: true
}
}
dest_uuid: "c2d38638ade14167b99bd0ceb219791e"
cas_config_opid_index: -1
I20260810 18:42:15.620704 14334 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: NON_VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: true } }
I20260810 18:42:15.624706 15195 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 882afacf744b4b6b89526c8f0108387e (127.4.228.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: NON_VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: true } } }
I20260810 18:42:15.625702 15093 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:42:15.628537 15093 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [term 1 FOLLOWER]: Refusing update from remote peer c2d38638ade14167b99bd0ceb219791e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260810 18:42:15.629106 15195 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.4.228.253:39135
I20260810 18:42:15.629663 15195 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c2d38638ade14167b99bd0ceb219791e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: NON_VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: true } } }
I20260810 18:42:15.629942 15195 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [sys.catalog]: This master's current role is: LEADER
I20260810 18:42:15.630518 15034 master.cc:537] Master@127.4.228.253:39135 shutting down...
I20260810 18:42:15.630362 15196 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [LEADER]: Connected to new peer: Peer: permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: NON_VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.228.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.252.keytab.
Entry for principal oryx/127.4.228.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.252.keytab.
W20260810 18:42:15.634454 14309 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260810 18:42:15.635196 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.252:40375
--webserver_interface=127.4.228.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.252.keytab
--principal=oryx/127.4.228.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:36525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:45361,127.4.228.253:39135,127.4.228.252:40375
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:42:15.638104 14309 consensus_peers.cc:487] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e -> Peer 882afacf744b4b6b89526c8f0108387e (127.4.228.253:39135): Couldn't send request to peer 882afacf744b4b6b89526c8f0108387e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 18:42:15.648459 15034 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:42:15.648979 15034 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:42:15.649195 15034 tablet_replica.cc:330] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e: stopping tablet replica
I20260810 18:42:15.657579 15034 master.cc:552] Master@127.4.228.253:39135 shutdown complete.
I20260810 18:42:15.657745 15034 master_runner.cc:305] Clearing existing system tablet
I20260810 18:42:15.658269 15034 fs_manager.cc:359] Metadata directory not provided
I20260810 18:42:15.658380 15034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:42:15.666155 15034 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.002s sys 0.001s
I20260810 18:42:15.673990 15207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:15.674439 15034 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:42:15.674757 15034 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.000s sys 0.007s
I20260810 18:42:15.674860 15034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-1/wal
uuid: "882afacf744b4b6b89526c8f0108387e"
format_stamp: "Formatted at 2026-08-10 18:42:15 on dist-test-slave-7kzw"
server_key: "5877beb787714755e801041ad5eb01e9"
server_key_iv: "1f69e9221f56d42ad7fcf0f9f63f34db"
server_key_version: "encryptionkey@0"
I20260810 18:42:15.680694 15034 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:42:15.689462 15034 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 18:42:15.689687 15034 log.cc:1196] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260810 18:42:15.690167 15034 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e: Deleting consensus metadata
I20260810 18:42:15.690510 15034 master_runner.cc:315] Copying system tablet from 127.4.228.254:45361
I20260810 18:42:15.696492 15034 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e: tablet copy: Beginning tablet copy session from remote peer at address 127.4.228.254:45361
I20260810 18:42:15.722412 15148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:15.697818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38784 (local address 127.4.228.254:45361)
0810 18:42:15.698023 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:15.698030 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:15.705598 (+ 7568us) server_negotiation.cc:407] Connection header received
0810 18:42:15.705976 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:15.705983 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:15.706082 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:15.706163 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:15.707673 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:15.708848 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:15.710788 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:15.711548 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:15.711661 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:15.714673 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:15.714682 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:15.714687 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:15.714733 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:15.718736 (+ 4003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:15.719827 (+ 1091us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:15.719833 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:15.719837 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:15.720028 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:15.720558 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:15.720564 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:15.720567 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:15.720976 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:15.721155 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:15.721739 (+ 584us) server_negotiation.cc:299] Negotiation successful
0810 18:42:15.721991 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 18:42:15.722878 15214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:15.697685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.0.0.1:38784)
0810 18:42:15.705446 (+ 7761us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:15.705470 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:15.705756 (+ 286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:15.706540 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:15.706547 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:15.707114 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:15.707437 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:15.707452 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:15.709064 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:15.709070 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:15.710596 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:15.710624 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:15.712009 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:15.712015 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:15.712236 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:15.713103 (+ 867us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:15.713129 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:15.714355 (+ 1226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:15.718904 (+ 4549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:15.718910 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:15.718915 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:15.719666 (+ 751us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:15.720163 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:15.720169 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:15.720173 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:15.720413 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:15.721128 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:15.721134 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:15.721369 (+ 235us) client_negotiation.cc:769] Sending connection context
0810 18:42:15.721591 (+ 222us) client_negotiation.cc:240] Negotiation successful
0810 18:42:15.721982 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7576,"thread_start_us":7421,"threads_started":1}
I20260810 18:42:15.724336 14360 tablet_copy_service.cc:140] P c2d38638ade14167b99bd0ceb219791e: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 882afacf744b4b6b89526c8f0108387e ({username='oryx', principal='oryx/127.4.228.253@KRBTEST.COM'} at 127.0.0.1:38784)
I20260810 18:42:15.724854 14360 tablet_copy_service.cc:161] P c2d38638ade14167b99bd0ceb219791e: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 882afacf744b4b6b89526c8f0108387e at {username='oryx', principal='oryx/127.4.228.253@KRBTEST.COM'} at 127.0.0.1:38784: session id = 882afacf744b4b6b89526c8f0108387e-00000000000000000000000000000000
I20260810 18:42:15.730703 14360 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 18:42:15.734655 15034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 18:42:15.745744 15034 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e: tablet copy: Starting download of 0 data blocks...
I20260810 18:42:15.746281 15034 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e: tablet copy: Starting download of 1 WAL segments...
I20260810 18:42:15.752306 15034 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 18:42:15.759819 14360 tablet_copy_service.cc:342] P c2d38638ade14167b99bd0ceb219791e: Request end of tablet copy session 882afacf744b4b6b89526c8f0108387e-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.4.228.253@KRBTEST.COM'} at 127.0.0.1:38784
I20260810 18:42:15.760505 14360 tablet_copy_service.cc:433] P c2d38638ade14167b99bd0ceb219791e: ending tablet copy session 882afacf744b4b6b89526c8f0108387e-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 882afacf744b4b6b89526c8f0108387e
I20260810 18:42:15.776157 15034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:42:15.780196 15217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:15.784780 15218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:15.795416 15223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:42:15.804518 15034 server_base.cc:1034] running on GCE node
I20260810 18:42:15.805099 15034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:42:15.806018 15034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:42:15.807296 15034 hybrid_clock.cc:648] HybridClock initialized: now 1786387335807200 us; error 42 us; skew 500 ppm
I20260810 18:42:15.807566 15034 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:42:15 dist-test-slave-7kzw krb5kdc[14258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387335, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:42:15.811066 15034 init.cc:377] Logged in from keytab as oryx/127.4.228.253@KRBTEST.COM (short username oryx)
I20260810 18:42:15.816607 15034 webserver.cc:460] Webserver started at http://127.4.228.253:37255/ using document root <none> and password file <none>
I20260810 18:42:15.817390 15034 fs_manager.cc:359] Metadata directory not provided
I20260810 18:42:15.817582 15034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:42:15.821674 15034 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 18:42:15.825551 15231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:15.826339 15034 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:42:15.826663 15034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-1/wal
uuid: "882afacf744b4b6b89526c8f0108387e"
format_stamp: "Formatted at 2026-08-10 18:42:15 on dist-test-slave-7kzw"
server_key: "5877beb787714755e801041ad5eb01e9"
server_key_iv: "1f69e9221f56d42ad7fcf0f9f63f34db"
server_key_version: "encryptionkey@0"
I20260810 18:42:15.826929 15034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:42:15.861181 15034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:42:15.867789 15034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:42:15.891714 15026 heartbeater.cc:503] Master 127.4.228.254:45361 was elected leader, sending a full tablet report...
I20260810 18:42:15.992473 15034 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.253:39135
I20260810 18:42:15.992967 15315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.253:39135 every 8 connection(s)
I20260810 18:42:15.994234 15034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-1/data/info.pb
I20260810 18:42:16.008853 15316 sys_catalog.cc:263] Verifying existing consensus state
I20260810 18:42:16.015611 15316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e: Bootstrap starting.
I20260810 18:42:16.051167 15316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:42:16.052100 15316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e: Bootstrap complete.
I20260810 18:42:16.053841 15316 raft_consensus.cc:348] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: NON_VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: true } }
I20260810 18:42:16.054234 15316 raft_consensus.cc:729] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 882afacf744b4b6b89526c8f0108387e, State: Initialized, Role: LEARNER
I20260810 18:42:16.054716 15316 consensus_queue.cc:260] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: NON_VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: true } }
I20260810 18:42:16.056614 15316 sys_catalog.cc:564] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:42:16.056458 15325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: NON_VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: true } } }
I20260810 18:42:16.057446 15325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [sys.catalog]: This master's current role is: LEARNER
I20260810 18:42:16.081323 15341 catalog_manager.cc:1259] Loaded cluster ID: df523c3658514fd88ae4a2bfce9ed027
I20260810 18:42:16.081492 15341 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e: loading cluster ID for follower catalog manager: success
I20260810 18:42:16.086592 15341 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e: acquiring CA information for follower catalog manager: success
I20260810 18:42:16.091138 15341 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 18:42:16.154363 15343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:16.144618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:39135 (local address 127.0.0.1:38912)
0810 18:42:16.145163 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:16.145187 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:16.145351 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:16.146461 (+ 1110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:16.146509 (+ 48us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:16.146549 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:42:16.146929 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:16.146947 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.149027 (+ 2080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.149033 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:16.151108 (+ 2075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:16.151141 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.153397 (+ 2256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.153405 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:16.153604 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:16.153697 (+ 93us) client_negotiation.cc:769] Sending connection context
0810 18:42:16.153816 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 18:42:16.153990 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":704,"threads_started":1}
I20260810 18:42:16.154425 15345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:16.145112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38912 (local address 127.4.228.253:39135)
0810 18:42:16.145985 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:16.145991 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:16.146018 (+ 27us) server_negotiation.cc:407] Connection header received
0810 18:42:16.146096 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:16.146101 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:16.146231 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:16.146379 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:42:16.147147 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.148831 (+ 1684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.151334 (+ 2503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.153160 (+ 1826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.153304 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:16.153573 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:16.153859 (+ 286us) server_negotiation.cc:299] Negotiation successful
0810 18:42:16.154010 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":728,"thread_start_us":535,"threads_started":1}
I20260810 18:42:16.156366 15275 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260810 18:42:16.157255 15325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2d38638ade14167b99bd0ceb219791e. Latest consensus state: current_term: 1 leader_uuid: "c2d38638ade14167b99bd0ceb219791e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: NON_VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: true } } }
I20260810 18:42:16.157622 15325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [sys.catalog]: This master's current role is: LEARNER
W20260810 18:42:16.282056 15202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:42:16.282477 15202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:42:16.282670 15202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:42:16.290736 15202 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:42:16.290918 15202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:42:16.291035 15202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:42:16.291144 15202 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:42:16.291229 15202 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:42:16.302878 15202 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:36525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:45361,127.4.228.253:39135,127.4.228.252:40375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.252.keytab
--principal=oryx/127.4.228.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.252:40375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.228.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:42:16.304420 15202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:42:16.306027 15202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:42:16.319820 15351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:16.319942 15352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:16.319802 15354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:42:16.324595 15202 server_base.cc:1034] running on GCE node
I20260810 18:42:16.325690 15202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:42:16.327808 15202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:42:16.329092 15202 hybrid_clock.cc:648] HybridClock initialized: now 1786387336329018 us; error 60 us; skew 500 ppm
I20260810 18:42:16.329499 15202 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:42:16 dist-test-slave-7kzw krb5kdc[14258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387336, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:42:16.333076 15202 init.cc:377] Logged in from keytab as oryx/127.4.228.252@KRBTEST.COM (short username oryx)
I20260810 18:42:16.334898 15202 webserver.cc:460] Webserver started at http://127.4.228.252:34059/ using document root <none> and password file <none>
I20260810 18:42:16.335836 15202 fs_manager.cc:359] Metadata directory not provided
I20260810 18:42:16.335966 15202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:42:16.336334 15202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:42:16.339632 15202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-2/data/instance:
uuid: "fd3cfc2801be46d4b2c030180a0d9a72"
format_stamp: "Formatted at 2026-08-10 18:42:16 on dist-test-slave-7kzw"
server_key: "ba83fa0f3e8575050c7318d07da03a95"
server_key_iv: "5dccae6a857073d521551aac6e18b749"
server_key_version: "encryptionkey@0"
I20260810 18:42:16.340504 15202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-2/wal/instance:
uuid: "fd3cfc2801be46d4b2c030180a0d9a72"
format_stamp: "Formatted at 2026-08-10 18:42:16 on dist-test-slave-7kzw"
server_key: "ba83fa0f3e8575050c7318d07da03a95"
server_key_iv: "5dccae6a857073d521551aac6e18b749"
server_key_version: "encryptionkey@0"
I20260810 18:42:16.346364 15202 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260810 18:42:16.350450 15369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:16.352119 15202 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 18:42:16.352321 15202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-2/wal
uuid: "fd3cfc2801be46d4b2c030180a0d9a72"
format_stamp: "Formatted at 2026-08-10 18:42:16 on dist-test-slave-7kzw"
server_key: "ba83fa0f3e8575050c7318d07da03a95"
server_key_iv: "5dccae6a857073d521551aac6e18b749"
server_key_version: "encryptionkey@0"
I20260810 18:42:16.352540 15202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:42:16.367776 15202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:42:16.372565 15202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:42:16.372900 15202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:42:16.396220 15202 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.252:40375
I20260810 18:42:16.396219 15445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.252:40375 every 8 connection(s)
I20260810 18:42:16.397948 15202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-2/data/info.pb
I20260810 18:42:16.402423 15446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:42:16.407380 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 15202
I20260810 18:42:16.410571 15446 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:42:16 dist-test-slave-7kzw krb5kdc[14258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387336, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.252@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM
Aug 10 18:42:16 dist-test-slave-7kzw krb5kdc[14258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.228.252@KRBTEST.COM
I20260810 18:42:16.434028 15452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:16.414349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38786 (local address 127.4.228.254:45361)
0810 18:42:16.414913 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:16.414920 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:16.415477 (+ 557us) server_negotiation.cc:407] Connection header received
0810 18:42:16.415839 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:16.415845 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:16.415944 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:16.416121 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:16.417383 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.418252 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.420103 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.420850 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.420959 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:16.425434 (+ 4475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:16.425440 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:16.425443 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:16.425481 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:16.429327 (+ 3846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:16.430613 (+ 1286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:16.430619 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:16.430623 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:16.430810 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:16.431392 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:16.431398 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:16.431402 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:16.431831 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:16.432008 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:16.433296 (+ 1288us) server_negotiation.cc:299] Negotiation successful
0810 18:42:16.433515 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":286,"threads_started":1}
I20260810 18:42:16.435539 15453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:16.414230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.0.0.1:38786)
0810 18:42:16.415227 (+ 997us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:16.415273 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:16.415581 (+ 308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:16.416219 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:16.416224 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:16.416828 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:16.417201 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:16.417217 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.418416 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.418422 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:16.419929 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:16.419943 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.421107 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.421115 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:16.421349 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:16.422448 (+ 1099us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:16.422543 (+ 95us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:16.425194 (+ 2651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:16.429503 (+ 4309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:16.429521 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:16.429538 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:16.430264 (+ 726us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:16.430953 (+ 689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:16.430959 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:16.430963 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:16.431223 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:16.432014 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:16.432022 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:16.432583 (+ 561us) client_negotiation.cc:769] Sending connection context
0810 18:42:16.432983 (+ 400us) client_negotiation.cc:240] Negotiation successful
0810 18:42:16.433620 (+ 637us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":791,"mutex_wait_us":72,"thread_start_us":637,"threads_started":1}
I20260810 18:42:16.435516 15451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:16.410209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34614 (local address 127.4.228.252:40375)
0810 18:42:16.412107 (+ 1898us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:16.412125 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:16.412183 (+ 58us) server_negotiation.cc:407] Connection header received
0810 18:42:16.414424 (+ 2241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:16.414457 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:16.415155 (+ 698us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:16.415716 (+ 561us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:16.417535 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.418985 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.420687 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.421522 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.421666 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:16.427085 (+ 5419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:16.427130 (+ 45us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:16.427150 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:16.427196 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:16.431171 (+ 3975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:16.432228 (+ 1057us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:16.432238 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:16.432241 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:16.432428 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:16.432918 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:16.432925 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:16.432931 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:16.433455 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:16.433813 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:16.434571 (+ 758us) server_negotiation.cc:299] Negotiation successful
0810 18:42:16.434861 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":918,"thread_start_us":654,"threads_started":1}
I20260810 18:42:16.437399 15446 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:42:16 dist-test-slave-7kzw krb5kdc[14258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387336, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.252@KRBTEST.COM for oryx/127.4.228.253@KRBTEST.COM
I20260810 18:42:16.453248 15453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:16.438345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:39135 (local address 127.0.0.1:38914)
0810 18:42:16.438640 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:16.438665 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:16.438841 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:16.439222 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:16.439227 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:16.439737 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:16.439899 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:16.439909 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.441022 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.441029 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:16.442080 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:16.442093 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.443024 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.443031 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:16.443267 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:16.444029 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:16.444050 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:16.446381 (+ 2331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:16.450086 (+ 3705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:16.450093 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:16.450097 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:16.450738 (+ 641us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:16.451378 (+ 640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:16.451382 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:16.451385 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:16.451545 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:16.452316 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:16.452320 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:16.452474 (+ 154us) client_negotiation.cc:769] Sending connection context
0810 18:42:16.452611 (+ 137us) client_negotiation.cc:240] Negotiation successful
0810 18:42:16.452958 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260810 18:42:16.453629 15345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:16.438372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38914 (local address 127.4.228.253:39135)
0810 18:42:16.438666 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:16.438673 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:16.438764 (+ 91us) server_negotiation.cc:407] Connection header received
0810 18:42:16.438988 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:16.438999 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:16.439073 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:16.439195 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:16.440070 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.440868 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.442269 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.442825 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.442980 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:16.446673 (+ 3693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:16.446680 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:16.446684 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:16.446726 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:16.449880 (+ 3154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:16.451049 (+ 1169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:16.451054 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:16.451057 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:16.451241 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:16.451799 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:16.451804 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:16.451806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:16.452173 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:16.452323 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:16.452805 (+ 482us) server_negotiation.cc:299] Negotiation successful
0810 18:42:16.453024 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156}
I20260810 18:42:16.454885 15446 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.228.252" port: 40375 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:42:16 dist-test-slave-7kzw krb5kdc[14258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387336, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.252@KRBTEST.COM for oryx/127.4.228.252@KRBTEST.COM
I20260810 18:42:16.469413 15453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:16.455676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:40375 (local address 127.0.0.1:34626)
0810 18:42:16.455946 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:16.455969 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:16.456167 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:16.456527 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:16.456531 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:16.457150 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:16.457317 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:16.457326 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.458502 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.458510 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:16.459583 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:16.459598 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.460386 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.460392 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:16.460698 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:16.461359 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:16.461378 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:16.463748 (+ 2370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:16.466989 (+ 3241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:16.466995 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:16.466999 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:16.467379 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:16.467855 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:16.467859 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:16.467862 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:16.468038 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:16.468575 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:16.468579 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:16.468729 (+ 150us) client_negotiation.cc:769] Sending connection context
0810 18:42:16.468862 (+ 133us) client_negotiation.cc:240] Negotiation successful
0810 18:42:16.469081 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"mutex_wait_us":26}
I20260810 18:42:16.469776 15451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:16.455754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34626 (local address 127.4.228.252:40375)
0810 18:42:16.455947 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:16.455955 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:16.456105 (+ 150us) server_negotiation.cc:407] Connection header received
0810 18:42:16.456313 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:16.456319 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:16.456396 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:16.456507 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:16.457466 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.458349 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.459765 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.460218 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.460375 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:16.463983 (+ 3608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:16.463988 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:16.463992 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:16.464022 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:16.466840 (+ 2818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:16.467511 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:16.467519 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:16.467523 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:16.467697 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:16.468152 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:16.468156 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:16.468158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:16.468456 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:16.468572 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:16.468979 (+ 407us) server_negotiation.cc:299] Negotiation successful
0810 18:42:16.469325 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":161,"server-negotiator.queue_time_us":69}
I20260810 18:42:16.475207 15446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72: Bootstrap starting.
I20260810 18:42:16.478655 15446 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72: Neither blocks nor log segments found. Creating new log.
I20260810 18:42:16.479776 15446 log.cc:824] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72: Log is configured to *not* fsync() on all Append() calls
I20260810 18:42:16.483208 15446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72: No bootstrap required, opened a new log
I20260810 18:42:16.492712 15446 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 40375 } }
I20260810 18:42:16.493196 15446 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:42:16.493358 15446 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd3cfc2801be46d4b2c030180a0d9a72, State: Initialized, Role: FOLLOWER
I20260810 18:42:16.494103 15446 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 40375 } }
I20260810 18:42:16.495262 15456 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 40375 } } }
I20260810 18:42:16.495606 15456 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:42:16.496682 15446 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 18:42:16.507773 15473 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:42:16.507961 15473 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:42:16.514088 15202 master_runner.cc:428] Detected that this master fd3cfc2801be46d4b2c030180a0d9a72 is joining an existing cluster
I20260810 18:42:16.514263 15202 master_runner.cc:432] Initiating AddMaster RPC to add 127.4.228.252:40375
I20260810 18:42:16.541313 15452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:16.519200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38790 (local address 127.4.228.254:45361)
0810 18:42:16.519444 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:16.519454 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:16.521167 (+ 1713us) server_negotiation.cc:407] Connection header received
0810 18:42:16.521485 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:16.521496 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:16.521620 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:16.521715 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:16.525729 (+ 4014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.526992 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.529318 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.529993 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.530103 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:16.533823 (+ 3720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:16.533830 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:16.533836 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:16.533882 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:16.537596 (+ 3714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:16.538680 (+ 1084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:16.538687 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:16.538690 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:16.538857 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:16.539379 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:16.539383 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:16.539386 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:16.539818 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:16.539972 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:16.540635 (+ 663us) server_negotiation.cc:299] Negotiation successful
0810 18:42:16.540910 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260810 18:42:16.541836 15345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:16.519485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38930 (local address 127.4.228.253:39135)
0810 18:42:16.519754 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:16.519774 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:16.522889 (+ 3115us) server_negotiation.cc:407] Connection header received
0810 18:42:16.523120 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:16.523138 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:16.523243 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:16.523351 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:16.525929 (+ 2578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.527169 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.530096 (+ 2927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.530857 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.530974 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:16.534512 (+ 3538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:16.534520 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:16.534526 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:16.534573 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:16.538281 (+ 3708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:16.539338 (+ 1057us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:16.539347 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:16.539354 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:16.539526 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:16.540029 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:16.540033 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:16.540036 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:16.540406 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:16.540530 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:16.541212 (+ 682us) server_negotiation.cc:299] Negotiation successful
0810 18:42:16.541492 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260810 18:42:16.542650 15451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:16.521229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34628 (local address 127.4.228.252:40375)
0810 18:42:16.521423 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:16.521447 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:16.524964 (+ 3517us) server_negotiation.cc:407] Connection header received
0810 18:42:16.525201 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:16.525206 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:16.525268 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:16.525354 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:16.526850 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.528056 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.530118 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.530914 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.531026 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:16.533843 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:16.533849 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:16.533854 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:16.533889 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:16.539146 (+ 5257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:16.540042 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:16.540048 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:16.540052 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:16.540260 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:16.540722 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:16.540727 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:16.540731 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:16.541187 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:16.541317 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:16.541888 (+ 571us) server_negotiation.cc:299] Negotiation successful
0810 18:42:16.542140 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":53,"spinlock_wait_cycles":1920}
I20260810 18:42:16.543068 15492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:16.519940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:39135 (local address 127.0.0.1:38930)
0810 18:42:16.522738 (+ 2798us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:16.522761 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:16.522951 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:16.523455 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:16.523466 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:16.524976 (+ 1510us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:16.525681 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:16.525707 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.527437 (+ 1730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.527443 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:16.529900 (+ 2457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:16.529916 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.531313 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.531320 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:16.531701 (+ 381us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:16.532627 (+ 926us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:16.532657 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:16.534251 (+ 1594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:16.538553 (+ 4302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:16.538560 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:16.538567 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:16.539182 (+ 615us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:16.539690 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:16.539695 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:16.539698 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:16.539897 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:16.540622 (+ 725us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:16.540626 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:16.540861 (+ 235us) client_negotiation.cc:769] Sending connection context
0810 18:42:16.542102 (+ 1241us) client_negotiation.cc:240] Negotiation successful
0810 18:42:16.542529 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2683,"mutex_wait_us":198,"spinlock_wait_cycles":91136,"thread_start_us":404,"threads_started":1}
I20260810 18:42:16.544152 15491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:16.519216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.0.0.1:38790)
0810 18:42:16.520773 (+ 1557us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:16.520806 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:16.521252 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:16.522038 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:16.522046 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:16.524981 (+ 2935us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:16.525513 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:16.525533 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.527222 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.527226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:16.529099 (+ 1873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:16.529113 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.530441 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.530447 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:16.530717 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:16.531547 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:16.531575 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:16.533548 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:16.537799 (+ 4251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:16.537806 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:16.537811 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:16.538512 (+ 701us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:16.539034 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:16.539039 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:16.539043 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:16.539240 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:16.540049 (+ 809us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:16.540053 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:16.540324 (+ 271us) client_negotiation.cc:769] Sending connection context
0810 18:42:16.542150 (+ 1826us) client_negotiation.cc:240] Negotiation successful
0810 18:42:16.543738 (+ 1588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1337,"mutex_wait_us":313,"thread_start_us":4791,"threads_started":1}
I20260810 18:42:16.544166 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 14424
I20260810 18:42:16.546710 15490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:16.520931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:40375 (local address 127.0.0.1:34628)
0810 18:42:16.524640 (+ 3709us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:16.524662 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:16.525057 (+ 395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:16.525564 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:16.525571 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:16.526321 (+ 750us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:16.526688 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:16.526702 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.528363 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.528372 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:16.529959 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:16.529974 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.531362 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.531379 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:16.531694 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:16.532439 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:16.532468 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:16.533657 (+ 1189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:16.539327 (+ 5670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:16.539332 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:16.539337 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:16.539890 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:16.540419 (+ 529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:16.540425 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:16.540434 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:16.540587 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:16.541414 (+ 827us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:16.541419 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:16.541600 (+ 181us) client_negotiation.cc:769] Sending connection context
0810 18:42:16.542937 (+ 1337us) client_negotiation.cc:240] Negotiation successful
0810 18:42:16.546108 (+ 3171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3529,"mutex_wait_us":2856,"thread_start_us":1445,"threads_started":1}
Aug 10 18:42:16 dist-test-slave-7kzw krb5kdc[14258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387332, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.254@KRBTEST.COM for oryx/127.4.228.252@KRBTEST.COM
Aug 10 18:42:16 dist-test-slave-7kzw krb5kdc[14258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.228.193@KRBTEST.COM
I20260810 18:42:16.567792 15451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:16.552527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34642 (local address 127.4.228.252:40375)
0810 18:42:16.552737 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:16.552744 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:16.552765 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:42:16.552902 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:16.552908 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:16.553006 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:16.553122 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:16.554036 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.554907 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.556364 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.557071 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.557200 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:16.560711 (+ 3511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:16.560717 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:16.560720 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:16.560753 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:16.564616 (+ 3863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:16.565455 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:16.565459 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:16.565462 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:16.565589 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:16.566040 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:16.566046 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:16.566049 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:16.566423 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:16.566634 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:16.567177 (+ 543us) server_negotiation.cc:299] Negotiation successful
0810 18:42:16.567461 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 18:42:16.567828 15343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:16.551768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:40375 (local address 127.0.0.1:34642)
0810 18:42:16.552165 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:16.552189 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:16.552307 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:16.553182 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:16.553188 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:16.553705 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:16.553865 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:16.553875 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.555050 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.555056 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:16.556175 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:16.556192 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.557485 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.557491 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:16.557622 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:16.558261 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:16.558280 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:16.560500 (+ 2220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:16.564759 (+ 4259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:16.564767 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:16.564771 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:16.565285 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:16.565725 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:16.565730 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:16.565734 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:16.565918 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:16.566590 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:16.566597 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:16.566835 (+ 238us) client_negotiation.cc:769] Sending connection context
0810 18:42:16.567036 (+ 201us) client_negotiation.cc:240] Negotiation successful
0810 18:42:16.567405 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245}
I20260810 18:42:16.569645 14334 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.4.228.252:40375: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72"
member_type: NON_VOTER
last_known_addr {
host: "127.4.228.252"
port: 40375
}
attrs {
promote: true
}
}
dest_uuid: "c2d38638ade14167b99bd0ceb219791e"
cas_config_opid_index: 5
I20260810 18:42:16.570829 14334 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: NON_VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: true } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: NON_VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: true } }
I20260810 18:42:16.572916 15494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:16.545773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36868 (local address 127.4.228.193:41347)
0810 18:42:16.549361 (+ 3588us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:16.549371 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:16.549407 (+ 36us) server_negotiation.cc:407] Connection header received
0810 18:42:16.549487 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:16.549498 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:16.549894 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:16.550208 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:16.552031 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.553425 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.555722 (+ 2297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.556757 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.556946 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:16.564109 (+ 7163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:16.564138 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:16.564165 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:16.564224 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:16.568748 (+ 4524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:16.569728 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:16.569741 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:16.569745 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:16.569953 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:16.570531 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:16.570538 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:16.570541 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:16.571075 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:16.571447 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:16.571851 (+ 404us) server_negotiation.cc:299] Negotiation successful
0810 18:42:16.572386 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3368,"thread_start_us":3187,"threads_started":1}
I20260810 18:42:16.573454 15497 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER fd3cfc2801be46d4b2c030180a0d9a72 (127.4.228.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: NON_VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: true } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: NON_VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: true } } }
I20260810 18:42:16.574201 15275 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [term 1 LEARNER]: Refusing update from remote peer c2d38638ade14167b99bd0ceb219791e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 18:42:16.575747 15405 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:42:16.576938 15342 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [LEADER]: Connected to new peer: Peer: permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: NON_VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260810 18:42:16.578003 15497 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.4.228.252:40375
I20260810 18:42:16.578853 14605 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:42:16.579394 15342 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c2d38638ade14167b99bd0ceb219791e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: NON_VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: true } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: NON_VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: true } } }
I20260810 18:42:16.579874 15342 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [sys.catalog]: This master's current role is: LEADER
I20260810 18:42:16.581658 15275 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER fd3cfc2801be46d4b2c030180a0d9a72 (127.4.228.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: NON_VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: true } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: NON_VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: true } } }
I20260810 18:42:16.583235 15202 master.cc:537] Master@127.4.228.252:40375 shutting down...
I20260810 18:42:16.588222 15325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c2d38638ade14167b99bd0ceb219791e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: NON_VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: true } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: NON_VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: true } } }
I20260810 18:42:16.588805 15325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [sys.catalog]: This master's current role is: LEARNER
I20260810 18:42:16.589490 15342 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e: attempting to promote NON_VOTER 882afacf744b4b6b89526c8f0108387e to VOTER
I20260810 18:42:16.590989 15342 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: false } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: NON_VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: true } }
I20260810 18:42:16.593986 15275 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [term 1 LEARNER]: Refusing update from remote peer c2d38638ade14167b99bd0ceb219791e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260810 18:42:16.594977 15496 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [LEADER]: Connected to new peer: Peer: permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260810 18:42:16.598176 15497 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 882afacf744b4b6b89526c8f0108387e (127.4.228.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: false } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: NON_VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: true } } }
I20260810 18:42:16.599130 15275 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 882afacf744b4b6b89526c8f0108387e (127.4.228.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: false } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: NON_VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: true } } }
I20260810 18:42:16.604597 15496 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c2d38638ade14167b99bd0ceb219791e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: false } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: NON_VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: true } } }
I20260810 18:42:16.604984 15325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c2d38638ade14167b99bd0ceb219791e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: false } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: NON_VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: true } } }
I20260810 18:42:16.605293 15325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:42:16.605358 15496 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [sys.catalog]: This master's current role is: LEADER
I20260810 18:42:16.605923 15497 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c2d38638ade14167b99bd0ceb219791e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: false } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: NON_VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: true } } }
I20260810 18:42:16.606395 15497 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [sys.catalog]: This master's current role is: LEADER
I20260810 18:42:16.620348 15405 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [term 1 FOLLOWER]: Refusing update from remote peer c2d38638ade14167b99bd0ceb219791e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
W20260810 18:42:16.626132 15378 tablet.cc:2369] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:42:16.627255 14309 consensus_peers.cc:487] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e -> Peer fd3cfc2801be46d4b2c030180a0d9a72 (127.4.228.252:40375): Couldn't send request to peer fd3cfc2801be46d4b2c030180a0d9a72. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 18:42:16.633956 15202 raft_consensus.cc:2229] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:42:16.634531 15202 raft_consensus.cc:2258] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:42:16.634835 15202 tablet_replica.cc:330] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72: stopping tablet replica
I20260810 18:42:16.643136 15202 master.cc:552] Master@127.4.228.252:40375 shutdown complete.
I20260810 18:42:16.643257 15202 master_runner.cc:305] Clearing existing system tablet
I20260810 18:42:16.643697 15202 fs_manager.cc:359] Metadata directory not provided
I20260810 18:42:16.643829 15202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:42:16.647249 15202 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260810 18:42:16.649132 15507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:16.649559 15202 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:42:16.649850 15202 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260810 18:42:16.649962 15202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-2/wal
uuid: "fd3cfc2801be46d4b2c030180a0d9a72"
format_stamp: "Formatted at 2026-08-10 18:42:16 on dist-test-slave-7kzw"
server_key: "ba83fa0f3e8575050c7318d07da03a95"
server_key_iv: "5dccae6a857073d521551aac6e18b749"
server_key_version: "encryptionkey@0"
I20260810 18:42:16.655556 15202 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:42:16.663950 15202 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 18:42:16.664202 15202 log.cc:1196] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260810 18:42:16.664862 15202 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72: Deleting consensus metadata
I20260810 18:42:16.665172 15202 master_runner.cc:315] Copying system tablet from 127.4.228.254:45361
I20260810 18:42:16.668365 15202 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72: tablet copy: Beginning tablet copy session from remote peer at address 127.4.228.254:45361
I20260810 18:42:16.687820 15517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:16.669388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.0.0.1:38798)
0810 18:42:16.669898 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:16.669925 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:16.670080 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:16.670649 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:16.670654 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:16.671224 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:16.671477 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:16.671489 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.673453 (+ 1964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.673459 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:16.674952 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:16.674973 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.676398 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.676405 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:16.676574 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:16.677437 (+ 863us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:16.677462 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:16.678679 (+ 1217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:16.684184 (+ 5505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:16.684190 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:16.684195 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:16.684796 (+ 601us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:16.685284 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:16.685289 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:16.685293 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:16.685529 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:16.686410 (+ 881us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:16.686416 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:16.686730 (+ 314us) client_negotiation.cc:769] Sending connection context
0810 18:42:16.686906 (+ 176us) client_negotiation.cc:240] Negotiation successful
0810 18:42:16.687313 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":304,"threads_started":1}
I20260810 18:42:16.688251 15452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:16.669470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38798 (local address 127.4.228.254:45361)
0810 18:42:16.669843 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:16.669849 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:16.670029 (+ 180us) server_negotiation.cc:407] Connection header received
0810 18:42:16.670265 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:16.670271 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:16.670378 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:16.670617 (+ 239us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:16.671666 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.673278 (+ 1612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.675142 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:16.676199 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:16.676345 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:16.678929 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:16.678937 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:16.678942 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:16.678986 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:16.683995 (+ 5009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:16.684951 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:16.684958 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:16.684961 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:16.685137 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:16.685679 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:16.685688 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:16.685694 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:16.686269 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:16.686428 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:16.687105 (+ 677us) server_negotiation.cc:299] Negotiation successful
0810 18:42:16.687377 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243}
I20260810 18:42:16.688874 14360 tablet_copy_service.cc:140] P c2d38638ade14167b99bd0ceb219791e: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer fd3cfc2801be46d4b2c030180a0d9a72 ({username='oryx', principal='oryx/127.4.228.252@KRBTEST.COM'} at 127.0.0.1:38798)
I20260810 18:42:16.689076 14360 tablet_copy_service.cc:161] P c2d38638ade14167b99bd0ceb219791e: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer fd3cfc2801be46d4b2c030180a0d9a72 at {username='oryx', principal='oryx/127.4.228.252@KRBTEST.COM'} at 127.0.0.1:38798: session id = fd3cfc2801be46d4b2c030180a0d9a72-00000000000000000000000000000000
I20260810 18:42:16.692800 14360 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 18:42:16.695427 15202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 18:42:16.705120 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 14424
I20260810 18:42:16.706612 15202 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72: tablet copy: Starting download of 0 data blocks...
I20260810 18:42:16.707022 15202 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72: tablet copy: Starting download of 1 WAL segments...
I20260810 18:42:16.715893 15202 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 18:42:16.722107 14360 tablet_copy_service.cc:342] P c2d38638ade14167b99bd0ceb219791e: Request end of tablet copy session fd3cfc2801be46d4b2c030180a0d9a72-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.4.228.252@KRBTEST.COM'} at 127.0.0.1:38798
I20260810 18:42:16.722306 14360 tablet_copy_service.cc:433] P c2d38638ade14167b99bd0ceb219791e: ending tablet copy session fd3cfc2801be46d4b2c030180a0d9a72-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer fd3cfc2801be46d4b2c030180a0d9a72
I20260810 18:42:16.737756 15202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 18:42:16.739177 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.193:41347
--local_ip_for_outbound_sockets=127.4.228.193
--tserver_master_addrs=127.4.228.254:45361
--webserver_port=41307
--webserver_interface=127.4.228.193
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.193.keytab
--principal=oryx/127.4.228.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:36525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.4.228.254:45361,127.4.228.253:39135
--tserver_master_addrs=127.4.228.254:45361,127.4.228.253:39135,127.4.228.252:40375 with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:42:16.740790 15518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:16.751241 15520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:16.757899 15524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:42:16.758302 15202 server_base.cc:1034] running on GCE node
I20260810 18:42:16.758946 15202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:42:16.759775 15202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:42:16.761180 15202 hybrid_clock.cc:648] HybridClock initialized: now 1786387336761120 us; error 46 us; skew 500 ppm
I20260810 18:42:16.761510 15202 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:42:16 dist-test-slave-7kzw krb5kdc[14258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387336, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:42:16.764850 15202 init.cc:377] Logged in from keytab as oryx/127.4.228.252@KRBTEST.COM (short username oryx)
I20260810 18:42:16.766513 15202 webserver.cc:460] Webserver started at http://127.4.228.252:46871/ using document root <none> and password file <none>
I20260810 18:42:16.766906 15202 fs_manager.cc:359] Metadata directory not provided
I20260810 18:42:16.767023 15202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:42:16.770753 15202 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.004s
I20260810 18:42:16.773067 15537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:16.773607 15202 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 18:42:16.773753 15202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-2/wal
uuid: "fd3cfc2801be46d4b2c030180a0d9a72"
format_stamp: "Formatted at 2026-08-10 18:42:16 on dist-test-slave-7kzw"
server_key: "ba83fa0f3e8575050c7318d07da03a95"
server_key_iv: "5dccae6a857073d521551aac6e18b749"
server_key_version: "encryptionkey@0"
I20260810 18:42:16.773972 15202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:42:16.803102 15202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:42:16.807996 15202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:42:16.836201 15202 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.252:40375
I20260810 18:42:16.836285 15622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.252:40375 every 8 connection(s)
I20260810 18:42:16.837546 15202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/master-2/data/info.pb
I20260810 18:42:16.843604 15623 sys_catalog.cc:263] Verifying existing consensus state
I20260810 18:42:16.848760 15623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72: Bootstrap starting.
I20260810 18:42:16.872114 15623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:42:16.873168 15623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72: Bootstrap complete.
I20260810 18:42:16.874794 15623 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: false } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: NON_VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: true } }
I20260810 18:42:16.875252 15623 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: fd3cfc2801be46d4b2c030180a0d9a72, State: Initialized, Role: LEARNER
I20260810 18:42:16.875582 15623 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: false } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: NON_VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: true } }
I20260810 18:42:16.876546 15628 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: false } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: NON_VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: true } } }
I20260810 18:42:16.877085 15628 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [sys.catalog]: This master's current role is: LEARNER
I20260810 18:42:16.877169 15623 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:42:16.898346 15642 catalog_manager.cc:1259] Loaded cluster ID: df523c3658514fd88ae4a2bfce9ed027
I20260810 18:42:16.898509 15642 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72: loading cluster ID for follower catalog manager: success
I20260810 18:42:16.902501 15642 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72: acquiring CA information for follower catalog manager: success
I20260810 18:42:16.905272 15642 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 18:42:17.025624 15343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:17.015947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:40375 (local address 127.0.0.1:34652)
0810 18:42:17.016314 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:17.016341 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:17.016530 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:17.017705 (+ 1175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:17.017714 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:17.017770 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:42:17.018079 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:17.018098 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:17.020692 (+ 2594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:17.020700 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:17.022689 (+ 1989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:17.022713 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:17.024702 (+ 1989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:17.024710 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:17.024925 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:17.024967 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 18:42:17.025090 (+ 123us) client_negotiation.cc:240] Negotiation successful
0810 18:42:17.025266 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186}
I20260810 18:42:17.025730 15644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:17.016239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34652 (local address 127.4.228.252:40375)
0810 18:42:17.017199 (+ 960us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:17.017206 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:17.017236 (+ 30us) server_negotiation.cc:407] Connection header received
0810 18:42:17.017325 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:17.017331 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:17.017460 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:17.017625 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:42:17.018309 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:17.020460 (+ 2151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:17.022946 (+ 2486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:17.024446 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:17.024626 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:17.024957 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:17.025161 (+ 204us) server_negotiation.cc:299] Negotiation successful
0810 18:42:17.025338 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":744,"thread_start_us":520,"threads_started":1}
I20260810 18:42:17.026984 15584 raft_consensus.cc:1203] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260810 18:42:17.027854 15628 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2d38638ade14167b99bd0ceb219791e. Latest consensus state: current_term: 1 leader_uuid: "c2d38638ade14167b99bd0ceb219791e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: false } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: NON_VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: true } } }
I20260810 18:42:17.028178 15628 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [sys.catalog]: This master's current role is: LEARNER
W20260810 18:42:17.211258 15526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:42:17.211728 15526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:42:17.211928 15526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:42:17.221395 15526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:42:17.221588 15526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:42:17.221834 15526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.193
I20260810 18:42:17.235224 15526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:36525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.193.keytab
--principal=oryx/127.4.228.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.193:41347
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.228.193
--webserver_port=41307
--tserver_master_addrs=127.4.228.254:45361,127.4.228.253:39135,127.4.228.252:40375
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.193
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:42:17.237015 15526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:42:17.238848 15526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:42:17.258383 15654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:17.258371 15657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:17.259501 15653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:42:17.260872 15526 server_base.cc:1034] running on GCE node
I20260810 18:42:17.261902 15526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:42:17.263473 15526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:42:17.264840 15526 hybrid_clock.cc:648] HybridClock initialized: now 1786387337264644 us; error 96 us; skew 500 ppm
I20260810 18:42:17.265337 15526 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:42:17 dist-test-slave-7kzw krb5kdc[14258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387337, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:42:17.271068 15526 init.cc:377] Logged in from keytab as oryx/127.4.228.193@KRBTEST.COM (short username oryx)
I20260810 18:42:17.273617 15526 webserver.cc:460] Webserver started at http://127.4.228.193:41307/ using document root <none> and password file <none>
I20260810 18:42:17.274762 15526 fs_manager.cc:359] Metadata directory not provided
I20260810 18:42:17.275043 15526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:42:17.283377 15526 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.003s sys 0.004s
I20260810 18:42:17.287961 15667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:17.290074 15526 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.003s
I20260810 18:42:17.290673 15526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-0/wal
uuid: "6a9e3573e79244e681476d93147477ff"
format_stamp: "Formatted at 2026-08-10 18:42:13 on dist-test-slave-7kzw"
server_key: "30bf8fc09bfe0a9bf65f924618efd682"
server_key_iv: "21af14d65a8f388955ff873164a8ec82"
server_key_version: "encryptionkey@0"
I20260810 18:42:17.292800 15526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:42:17.319264 15526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:42:17.325683 15526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:42:17.326076 15526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:42:17.327857 15526 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:42:17.330338 15526 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:42:17.330590 15526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:17.330770 15526 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:42:17.330893 15526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:17.384322 15526 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.193:41347
I20260810 18:42:17.384490 15847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.193:41347 every 8 connection(s)
I20260810 18:42:17.386852 15526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
I20260810 18:42:17.393378 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 15526
I20260810 18:42:17.393923 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 14621
Aug 10 18:42:17 dist-test-slave-7kzw krb5kdc[14258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387337, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.252@KRBTEST.COM
Aug 10 18:42:17 dist-test-slave-7kzw krb5kdc[14258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387337, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.253@KRBTEST.COM
Aug 10 18:42:17 dist-test-slave-7kzw krb5kdc[14258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.228.194@KRBTEST.COM
Aug 10 18:42:17 dist-test-slave-7kzw krb5kdc[14258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387337, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM
I20260810 18:42:17.468235 15860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:17.423582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:39135 (local address 127.4.228.193:38489)
0810 18:42:17.427938 (+ 4356us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:17.427965 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:17.428121 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:17.429191 (+ 1070us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:17.429198 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:17.430116 (+ 918us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:17.430529 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:17.430546 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:17.436650 (+ 6104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:17.436657 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:17.438908 (+ 2251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:17.438929 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:17.445627 (+ 6698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:17.445636 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:17.445935 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:17.449650 (+ 3715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:17.449677 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:17.457716 (+ 8039us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:17.462409 (+ 4693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:17.462423 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:17.462447 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:17.463278 (+ 831us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:17.463848 (+ 570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:17.463853 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:17.463856 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:17.464202 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:17.465064 (+ 862us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:17.465073 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:17.465588 (+ 515us) client_negotiation.cc:769] Sending connection context
0810 18:42:17.465950 (+ 362us) client_negotiation.cc:240] Negotiation successful
0810 18:42:17.466613 (+ 663us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4178,"thread_start_us":3982,"threads_started":1}
I20260810 18:42:17.469540 15859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:17.396126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47122 (local address 127.4.228.194:44749)
0810 18:42:17.436969 (+ 40843us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:17.436981 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:17.437030 (+ 49us) server_negotiation.cc:407] Connection header received
0810 18:42:17.437108 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:17.437118 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:17.437467 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:17.437678 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:17.441976 (+ 4298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:17.444339 (+ 2363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:17.447381 (+ 3042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:17.448689 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:17.448838 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:17.459616 (+ 10778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:17.459638 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:17.459677 (+ 39us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:17.459738 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:17.464702 (+ 4964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:17.465848 (+ 1146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:17.465857 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:17.465860 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:17.466039 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:17.466564 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:17.466568 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:17.466571 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:17.467035 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:17.468607 (+ 1572us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:17.468808 (+ 201us) server_negotiation.cc:299] Negotiation successful
0810 18:42:17.469104 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40624,"thread_start_us":42033,"threads_started":1}
I20260810 18:42:17.469756 15862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:17.424093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:38489 (local address 127.4.228.253:39135)
0810 18:42:17.428603 (+ 4510us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:17.428612 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:17.428639 (+ 27us) server_negotiation.cc:407] Connection header received
0810 18:42:17.428707 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:17.428715 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:17.428818 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:17.428950 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:17.435162 (+ 6212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:17.436406 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:17.439166 (+ 2760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:17.439978 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:17.440143 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:17.458010 (+ 17867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:17.458019 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:17.458024 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:17.458075 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:17.462200 (+ 4125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:17.463438 (+ 1238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:17.463444 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:17.463448 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:17.463663 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:17.464382 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:17.464388 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:17.464391 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:17.464822 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:17.464963 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:17.469052 (+ 4089us) server_negotiation.cc:299] Negotiation successful
0810 18:42:17.469330 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4370,"thread_start_us":4264,"threads_started":1}
I20260810 18:42:17.471268 14817 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:42:17.472704 15850 heartbeater.cc:347] Connected to a master server at 127.4.228.253:39135
I20260810 18:42:17.473198 15850 heartbeater.cc:464] Registering TS with master...
I20260810 18:42:17.474540 15850 heartbeater.cc:511] Master 127.4.228.253:39135 requested a full tablet report, sending...
I20260810 18:42:17.476382 15853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:17.390856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:60853 (local address 127.4.228.254:45361)
0810 18:42:17.441366 (+ 50510us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:17.441373 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:17.441394 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:42:17.441457 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:17.441462 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:17.441585 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:17.441713 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:17.450628 (+ 8915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:17.451643 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:17.454087 (+ 2444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:17.455281 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:17.457967 (+ 2686us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:17.462040 (+ 4073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:17.462050 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:17.462057 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:17.462115 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:17.473068 (+ 10953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:17.473880 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:17.473886 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:17.473889 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:17.474095 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:17.474607 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:17.474612 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:17.474615 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:17.474909 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:17.475069 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:17.475748 (+ 679us) server_negotiation.cc:299] Negotiation successful
0810 18:42:17.475979 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50366,"thread_start_us":53621,"threads_started":1}
I20260810 18:42:17.476473 15855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:17.391872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.4.228.193:60853)
0810 18:42:17.420260 (+ 28388us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:17.420291 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:17.423081 (+ 2790us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:17.441738 (+ 18657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:17.441742 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:17.442859 (+ 1117us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:17.443819 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:17.443838 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:17.451814 (+ 7976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:17.451820 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:17.453884 (+ 2064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:17.453906 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:17.457208 (+ 3302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:17.457219 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:17.457542 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:17.458307 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:17.458339 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:17.461709 (+ 3370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:17.473247 (+ 11538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:17.473252 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:17.473256 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:17.473752 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:17.474239 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:17.474251 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:17.474255 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:17.474436 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:17.475049 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:17.475054 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:17.475344 (+ 290us) client_negotiation.cc:769] Sending connection context
0810 18:42:17.475514 (+ 170us) client_negotiation.cc:240] Negotiation successful
0810 18:42:17.475870 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28150,"mutex_wait_us":35,"thread_start_us":26433,"threads_started":1}
I20260810 18:42:17.477444 15849 heartbeater.cc:347] Connected to a master server at 127.4.228.254:45361
I20260810 18:42:17.477666 15849 heartbeater.cc:464] Registering TS with master...
I20260810 18:42:17.478183 15849 heartbeater.cc:511] Master 127.4.228.254:45361 requested a full tablet report, sending...
I20260810 18:42:17.478597 15253 ts_manager.cc:194] Registered new tserver with Master: 6a9e3573e79244e681476d93147477ff (127.4.228.193:41347)
I20260810 18:42:17.479503 14334 ts_manager.cc:194] Re-registered known tserver with Master: 6a9e3573e79244e681476d93147477ff (127.4.228.193:41347)
I20260810 18:42:17.480306 14334 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.228.193@KRBTEST.COM'} at 127.4.228.193:60853
I20260810 18:42:17.481978 15644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:17.393383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:49903 (local address 127.4.228.252:40375)
0810 18:42:17.393651 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:17.393657 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:17.416963 (+ 23306us) server_negotiation.cc:407] Connection header received
0810 18:42:17.423750 (+ 6787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:17.423761 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:17.423881 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:17.423970 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:17.430764 (+ 6794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:17.432020 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:17.440437 (+ 8417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:17.441224 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:17.441336 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:17.463475 (+ 22139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:17.463485 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:17.463493 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:17.463554 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:17.468168 (+ 4614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:17.470290 (+ 2122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:17.470299 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:17.470305 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:17.470897 (+ 592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:17.471519 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:17.471527 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:17.471534 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:17.472037 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:17.472240 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:17.481288 (+ 9048us) server_negotiation.cc:299] Negotiation successful
0810 18:42:17.481614 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260810 18:42:17.482641 15856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:17.391872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:40375 (local address 127.4.228.193:49903)
0810 18:42:17.416544 (+ 24672us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:17.416644 (+ 100us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:17.423547 (+ 6903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:17.424614 (+ 1067us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:17.424630 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:17.426020 (+ 1390us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:17.427205 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:17.427242 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:17.432277 (+ 5035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:17.432286 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:17.434913 (+ 2627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:17.434933 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:17.444107 (+ 9174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:17.444118 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:17.444604 (+ 486us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:17.446869 (+ 2265us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:17.446926 (+ 57us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:17.456826 (+ 9900us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:17.468858 (+ 12032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:17.468867 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:17.468872 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:17.469491 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:17.471076 (+ 1585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:17.471083 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:17.471088 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:17.471290 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:17.472355 (+ 1065us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:17.472363 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:17.480837 (+ 8474us) client_negotiation.cc:769] Sending connection context
0810 18:42:17.481109 (+ 272us) client_negotiation.cc:240] Negotiation successful
0810 18:42:17.481623 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24078,"thread_start_us":28418,"threads_started":1}
I20260810 18:42:17.484179 15848 heartbeater.cc:347] Connected to a master server at 127.4.228.252:40375
I20260810 18:42:17.484305 15848 heartbeater.cc:464] Registering TS with master...
I20260810 18:42:17.485037 15848 heartbeater.cc:511] Master 127.4.228.252:40375 requested a full tablet report, sending...
I20260810 18:42:17.487505 15567 ts_manager.cc:194] Registered new tserver with Master: 6a9e3573e79244e681476d93147477ff (127.4.228.193:41347)
I20260810 18:42:17.612419 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 14621
I20260810 18:42:17.620110 15496 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c2d38638ade14167b99bd0ceb219791e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: false } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: NON_VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: true } } }
I20260810 18:42:17.620500 15496 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [sys.catalog]: This master's current role is: LEADER
I20260810 18:42:17.635226 15864 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e: attempting to promote NON_VOTER fd3cfc2801be46d4b2c030180a0d9a72 to VOTER
I20260810 18:42:17.636366 15864 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: false } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: false } }
I20260810 18:42:17.639601 15275 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [term 1 FOLLOWER]: Refusing update from remote peer c2d38638ade14167b99bd0ceb219791e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260810 18:42:17.639639 15584 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [term 1 LEARNER]: Refusing update from remote peer c2d38638ade14167b99bd0ceb219791e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260810 18:42:17.640400 15864 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260810 18:42:17.640956 15496 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [LEADER]: Connected to new peer: Peer: permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260810 18:42:17.653244 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.194:44749
--local_ip_for_outbound_sockets=127.4.228.194
--tserver_master_addrs=127.4.228.254:45361
--webserver_port=43247
--webserver_interface=127.4.228.194
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.194.keytab
--principal=oryx/127.4.228.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:36525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.4.228.254:45361,127.4.228.253:39135
--tserver_master_addrs=127.4.228.254:45361,127.4.228.253:39135,127.4.228.252:40375 with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:42:17.657737 15866 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, fd3cfc2801be46d4b2c030180a0d9a72 (127.4.228.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: false } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: false } } }
I20260810 18:42:17.659158 15274 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, fd3cfc2801be46d4b2c030180a0d9a72 (127.4.228.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: false } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: false } } }
I20260810 18:42:17.663640 15496 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c2d38638ade14167b99bd0ceb219791e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: false } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: false } } }
I20260810 18:42:17.664003 15496 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [sys.catalog]: This master's current role is: LEADER
I20260810 18:42:17.664438 15496 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c2d38638ade14167b99bd0ceb219791e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: false } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: false } } }
I20260810 18:42:17.664690 15496 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [sys.catalog]: This master's current role is: LEADER
I20260810 18:42:17.664996 15496 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c2d38638ade14167b99bd0ceb219791e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: false } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: false } } }
I20260810 18:42:17.665243 15496 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e [sys.catalog]: This master's current role is: LEADER
I20260810 18:42:17.669977 15584 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, fd3cfc2801be46d4b2c030180a0d9a72 (127.4.228.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: false } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: false } } }
I20260810 18:42:17.672909 15878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c2d38638ade14167b99bd0ceb219791e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: false } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: false } } }
I20260810 18:42:17.673347 15878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:42:17.676839 15879 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c2d38638ade14167b99bd0ceb219791e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: false } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: false } } }
I20260810 18:42:17.677233 15879 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 18:42:18.089438 15876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:42:18.089846 15876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:42:18.090021 15876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:42:18.097786 15876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:42:18.097963 15876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:42:18.098136 15876 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.194
I20260810 18:42:18.109400 15876 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:36525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.194.keytab
--principal=oryx/127.4.228.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.194:44749
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.228.194
--webserver_port=43247
--tserver_master_addrs=127.4.228.254:45361,127.4.228.253:39135,127.4.228.252:40375
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.194
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:42:18.110980 15876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:42:18.112347 15876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:42:18.127522 15887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:18.127643 15891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:18.127928 15886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:42:18.128950 15876 server_base.cc:1034] running on GCE node
I20260810 18:42:18.129673 15876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:42:18.131067 15876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:42:18.132383 15876 hybrid_clock.cc:648] HybridClock initialized: now 1786387338132347 us; error 38 us; skew 500 ppm
I20260810 18:42:18.132789 15876 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:42:18 dist-test-slave-7kzw krb5kdc[14258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387338, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:42:18.136752 15876 init.cc:377] Logged in from keytab as oryx/127.4.228.194@KRBTEST.COM (short username oryx)
I20260810 18:42:18.138782 15876 webserver.cc:460] Webserver started at http://127.4.228.194:43247/ using document root <none> and password file <none>
I20260810 18:42:18.139776 15876 fs_manager.cc:359] Metadata directory not provided
I20260810 18:42:18.139917 15876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:42:18.146447 15876 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260810 18:42:18.150046 15906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:18.151702 15876 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:42:18.151921 15876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-1/wal
uuid: "70f6632aa17f433ab1c0bf5502ea03a1"
format_stamp: "Formatted at 2026-08-10 18:42:14 on dist-test-slave-7kzw"
server_key: "888ebb509af3963bab608fa001601a02"
server_key_iv: "38fdf4b3749a0f128395065ae2b04a68"
server_key_version: "encryptionkey@0"
I20260810 18:42:18.153014 15876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:42:18.171346 15876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:42:18.176998 15876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:42:18.177325 15876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:42:18.178676 15876 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:42:18.180177 15876 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:42:18.180269 15876 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:18.180348 15876 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:42:18.180409 15876 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:18.220414 15876 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.194:44749
I20260810 18:42:18.220479 16097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.194:44749 every 8 connection(s)
I20260810 18:42:18.222100 15876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
I20260810 18:42:18.227856 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 15876
I20260810 18:42:18.228296 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 14835
Aug 10 18:42:18 dist-test-slave-7kzw krb5kdc[14258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387338, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM
Aug 10 18:42:18 dist-test-slave-7kzw krb5kdc[14258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387338, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.252@KRBTEST.COM
Aug 10 18:42:18 dist-test-slave-7kzw krb5kdc[14258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.228.195@KRBTEST.COM
I20260810 18:42:18.272135 16105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:18.226209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:57415 (local address 127.4.228.254:45361)
0810 18:42:18.239548 (+ 13339us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:18.239556 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:18.239584 (+ 28us) server_negotiation.cc:407] Connection header received
0810 18:42:18.241747 (+ 2163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:18.241753 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:18.241862 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:18.242006 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:18.245749 (+ 3743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:18.247316 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:18.251092 (+ 3776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:18.252231 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:18.252355 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:18.259665 (+ 7310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:18.259674 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:18.259678 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:18.259734 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:18.265609 (+ 5875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:18.267306 (+ 1697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:18.267317 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:18.267323 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:18.267673 (+ 350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:18.268654 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:18.268672 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:18.268676 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:18.269261 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:18.269427 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:18.270979 (+ 1552us) server_negotiation.cc:299] Negotiation successful
0810 18:42:18.271431 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13178,"thread_start_us":13399,"threads_started":1}
Aug 10 18:42:18 dist-test-slave-7kzw krb5kdc[14258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387338, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.253@KRBTEST.COM
I20260810 18:42:18.273835 16108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:18.226818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.4.228.194:57415)
0810 18:42:18.238308 (+ 11490us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:18.238395 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:18.241391 (+ 2996us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:18.242423 (+ 1032us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:18.242443 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:18.243923 (+ 1480us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:18.245456 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:18.245480 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:18.247577 (+ 2097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:18.247585 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:18.250779 (+ 3194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:18.250802 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:18.252875 (+ 2073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:18.252895 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:18.253348 (+ 453us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:18.254425 (+ 1077us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:18.254485 (+ 60us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:18.259331 (+ 4846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:18.265895 (+ 6564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:18.265910 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:18.265934 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:18.267096 (+ 1162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:18.267894 (+ 798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:18.267900 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:18.267905 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:18.268452 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:18.269523 (+ 1071us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:18.269533 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:18.270298 (+ 765us) client_negotiation.cc:769] Sending connection context
0810 18:42:18.272421 (+ 2123us) client_negotiation.cc:240] Negotiation successful
0810 18:42:18.272848 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10946,"mutex_wait_us":148,"thread_start_us":13208,"threads_started":1}
I20260810 18:42:18.274858 16104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:18.226167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:50643 (local address 127.4.228.252:40375)
0810 18:42:18.240125 (+ 13958us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:18.240133 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:18.240776 (+ 643us) server_negotiation.cc:407] Connection header received
0810 18:42:18.242254 (+ 1478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:18.242259 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:18.242356 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:18.243071 (+ 715us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:18.245756 (+ 2685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:18.247429 (+ 1673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:18.251202 (+ 3773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:18.252288 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:18.252439 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:18.259705 (+ 7266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:18.259716 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:18.259724 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:18.259782 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:18.266168 (+ 6386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:18.267658 (+ 1490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:18.267665 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:18.267669 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:18.267905 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:18.268811 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:18.268827 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:18.268834 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:18.270113 (+ 1279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:18.270427 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:18.273810 (+ 3383us) server_negotiation.cc:299] Negotiation successful
0810 18:42:18.274096 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13794,"thread_start_us":13678,"threads_started":1}
I20260810 18:42:18.275451 16107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:18.226819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:40375 (local address 127.4.228.194:50643)
0810 18:42:18.240597 (+ 13778us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:18.240624 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:18.241430 (+ 806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:18.242575 (+ 1145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:18.242580 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:18.244040 (+ 1460us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:18.245436 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:18.245477 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:18.247700 (+ 2223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:18.247715 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:18.250975 (+ 3260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:18.250990 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:18.253002 (+ 2012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:18.253014 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:18.253363 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:18.255212 (+ 1849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:18.255250 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:18.259334 (+ 4084us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:18.266357 (+ 7023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:18.266362 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:18.266366 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:18.267464 (+ 1098us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:18.268122 (+ 658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:18.268127 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:18.268131 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:18.268454 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:18.270573 (+ 2119us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:18.270579 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:18.270839 (+ 260us) client_negotiation.cc:769] Sending connection context
0810 18:42:18.271138 (+ 299us) client_negotiation.cc:240] Negotiation successful
0810 18:42:18.271949 (+ 811us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13569,"mutex_wait_us":1386,"spinlock_wait_cycles":25216,"thread_start_us":14670,"threads_started":1}
I20260810 18:42:18.277431 16098 heartbeater.cc:347] Connected to a master server at 127.4.228.252:40375
I20260810 18:42:18.277503 16099 heartbeater.cc:347] Connected to a master server at 127.4.228.254:45361
I20260810 18:42:18.277920 16098 heartbeater.cc:464] Registering TS with master...
I20260810 18:42:18.278402 16099 heartbeater.cc:464] Registering TS with master...
I20260810 18:42:18.279877 16098 heartbeater.cc:511] Master 127.4.228.252:40375 requested a full tablet report, sending...
I20260810 18:42:18.280009 16099 heartbeater.cc:511] Master 127.4.228.254:45361 requested a full tablet report, sending...
I20260810 18:42:18.282384 14334 ts_manager.cc:194] Re-registered known tserver with Master: 70f6632aa17f433ab1c0bf5502ea03a1 (127.4.228.194:44749)
I20260810 18:42:18.282861 15567 ts_manager.cc:194] Registered new tserver with Master: 70f6632aa17f433ab1c0bf5502ea03a1 (127.4.228.194:44749)
I20260810 18:42:18.283149 16111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:18.230244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52774 (local address 127.4.228.195:40897)
0810 18:42:18.238143 (+ 7899us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:18.238156 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:18.247450 (+ 9294us) server_negotiation.cc:407] Connection header received
0810 18:42:18.248061 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:18.248073 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:18.248399 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:18.248549 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:18.252662 (+ 4113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:18.254591 (+ 1929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:18.256679 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:18.257539 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:18.257712 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:18.267781 (+ 10069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:18.267812 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:18.267838 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:18.267894 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:18.276675 (+ 8781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:18.278239 (+ 1564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:18.278253 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:18.278259 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:18.278638 (+ 379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:18.279392 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:18.279398 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:18.279401 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:18.280062 (+ 661us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:18.280639 (+ 577us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:18.281644 (+ 1005us) server_negotiation.cc:299] Negotiation successful
0810 18:42:18.282214 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7688,"thread_start_us":7486,"threads_started":1}
I20260810 18:42:18.284152 14334 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.228.194@KRBTEST.COM'} at 127.4.228.194:57415
I20260810 18:42:18.286007 16114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:18.248575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:39135 (local address 127.4.228.194:35503)
0810 18:42:18.256982 (+ 8407us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:18.257003 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:18.257142 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:18.258832 (+ 1690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:18.258842 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:18.259660 (+ 818us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:18.260337 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:18.260357 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:18.263075 (+ 2718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:18.263080 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:18.266095 (+ 3015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:18.266116 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:18.268110 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:18.268119 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:18.268507 (+ 388us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:18.269616 (+ 1109us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:18.269648 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:18.274096 (+ 4448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:18.281604 (+ 7508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:18.281610 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:18.281614 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:18.282372 (+ 758us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:18.283022 (+ 650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:18.283028 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:18.283034 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:18.283416 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:18.284381 (+ 965us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:18.284386 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:18.284626 (+ 240us) client_negotiation.cc:769] Sending connection context
0810 18:42:18.284806 (+ 180us) client_negotiation.cc:240] Negotiation successful
0810 18:42:18.285213 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5158,"thread_start_us":2208,"threads_started":1}
I20260810 18:42:18.286599 15020 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:42:18.287227 16113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:18.242565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:35503 (local address 127.4.228.253:39135)
0810 18:42:18.256246 (+ 13681us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:18.258102 (+ 1856us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:18.258136 (+ 34us) server_negotiation.cc:407] Connection header received
0810 18:42:18.258207 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:18.258213 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:18.258309 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:18.258580 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:18.260578 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:18.262783 (+ 2205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:18.266330 (+ 3547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:18.267492 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:18.267635 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:18.274565 (+ 6930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:18.274579 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:18.274587 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:18.274645 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:18.281371 (+ 6726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:18.282588 (+ 1217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:18.282594 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:18.282598 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:18.282859 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:18.283600 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:18.283606 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:18.283609 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:18.284086 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:18.284287 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:18.286409 (+ 2122us) server_negotiation.cc:299] Negotiation successful
0810 18:42:18.286714 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13509,"thread_start_us":15401,"threads_started":1}
I20260810 18:42:18.288954 16100 heartbeater.cc:347] Connected to a master server at 127.4.228.253:39135
I20260810 18:42:18.289217 16100 heartbeater.cc:464] Registering TS with master...
I20260810 18:42:18.290227 16100 heartbeater.cc:511] Master 127.4.228.253:39135 requested a full tablet report, sending...
I20260810 18:42:18.292160 15253 ts_manager.cc:194] Registered new tserver with Master: 70f6632aa17f433ab1c0bf5502ea03a1 (127.4.228.194:44749)
I20260810 18:42:18.379410 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 14835
I20260810 18:42:18.410023 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.195:40897
--local_ip_for_outbound_sockets=127.4.228.195
--tserver_master_addrs=127.4.228.254:45361
--webserver_port=34799
--webserver_interface=127.4.228.195
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.195.keytab
--principal=oryx/127.4.228.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:36525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.4.228.254:45361,127.4.228.253:39135
--tserver_master_addrs=127.4.228.254:45361,127.4.228.253:39135,127.4.228.252:40375 with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:42:18.483871 15849 heartbeater.cc:503] Master 127.4.228.254:45361 was elected leader, sending a full tablet report...
W20260810 18:42:18.781370 16117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:42:18.781737 16117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:42:18.781917 16117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:42:18.791167 16117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:42:18.791306 16117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:42:18.791508 16117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.195
I20260810 18:42:18.803951 16117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:36525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.195.keytab
--principal=oryx/127.4.228.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.195:40897
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.228.195
--webserver_port=34799
--tserver_master_addrs=127.4.228.254:45361,127.4.228.253:39135,127.4.228.252:40375
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.195
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:42:18.805634 16117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:42:18.807351 16117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:42:18.827731 16124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:18.827818 16125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:18.827685 16129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:42:18.829229 16117 server_base.cc:1034] running on GCE node
I20260810 18:42:18.830041 16117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:42:18.831266 16117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:42:18.832522 16117 hybrid_clock.cc:648] HybridClock initialized: now 1786387338832438 us; error 71 us; skew 500 ppm
I20260810 18:42:18.832973 16117 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:42:18 dist-test-slave-7kzw krb5kdc[14258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387338, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:42:18.842962 16117 init.cc:377] Logged in from keytab as oryx/127.4.228.195@KRBTEST.COM (short username oryx)
I20260810 18:42:18.844722 16117 webserver.cc:460] Webserver started at http://127.4.228.195:34799/ using document root <none> and password file <none>
I20260810 18:42:18.845571 16117 fs_manager.cc:359] Metadata directory not provided
I20260810 18:42:18.845737 16117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:42:18.852077 16117 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 18:42:18.855682 16136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:18.857738 16117 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:42:18.858040 16117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-2/wal
uuid: "b5068707091b42a5b99875e6389e158e"
format_stamp: "Formatted at 2026-08-10 18:42:14 on dist-test-slave-7kzw"
server_key: "a81f90666e805354117093ff9438679c"
server_key_iv: "7e85402bac5cb2526928474d981463fe"
server_key_version: "encryptionkey@0"
I20260810 18:42:18.859324 16117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:42:18.872783 16117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:42:18.878059 16117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:42:18.878490 16117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:42:18.880016 16117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:42:18.881619 16117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:42:18.881794 16117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:18.881882 16117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:42:18.881940 16117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:18.921303 16117 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.195:40897
I20260810 18:42:18.921406 16316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.195:40897 every 8 connection(s)
I20260810 18:42:18.922915 16117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
I20260810 18:42:18.931242 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 16117
Aug 10 18:42:18 dist-test-slave-7kzw krb5kdc[14258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387338, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.195@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM
Aug 10 18:42:18 dist-test-slave-7kzw krb5kdc[14258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387338, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.195@KRBTEST.COM for oryx/127.4.228.252@KRBTEST.COM
Aug 10 18:42:18 dist-test-slave-7kzw krb5kdc[14258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387338, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.195@KRBTEST.COM for oryx/127.4.228.253@KRBTEST.COM
I20260810 18:42:18.959600 16327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:18.929307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:40375 (local address 127.4.228.195:45191)
0810 18:42:18.930048 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:18.930071 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:18.931670 (+ 1599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:18.932728 (+ 1058us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:18.932734 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:18.934240 (+ 1506us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:18.935678 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:18.935701 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:18.937363 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:18.937373 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:18.939672 (+ 2299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:18.939684 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:18.941103 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:18.941114 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:18.941437 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:18.942204 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:18.942245 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:18.946453 (+ 4208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:18.951844 (+ 5391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:18.951856 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:18.951879 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:18.952885 (+ 1006us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:18.954240 (+ 1355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:18.954246 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:18.954250 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:18.954588 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:18.955705 (+ 1117us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:18.955724 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:18.956268 (+ 544us) client_negotiation.cc:769] Sending connection context
0810 18:42:18.956839 (+ 571us) client_negotiation.cc:240] Negotiation successful
0810 18:42:18.957490 (+ 651us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"mutex_wait_us":93,"spinlock_wait_cycles":102272,"thread_start_us":348,"threads_started":1}
I20260810 18:42:18.962040 16331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:18.929474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:45191 (local address 127.4.228.252:40375)
0810 18:42:18.930901 (+ 1427us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:18.930905 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:18.930922 (+ 17us) server_negotiation.cc:407] Connection header received
0810 18:42:18.932194 (+ 1272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:18.932200 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:18.932300 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:18.932404 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:18.935923 (+ 3519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:18.937143 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:18.939817 (+ 2674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:18.940559 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:18.940725 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:18.946832 (+ 6107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:18.946839 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:18.946844 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:18.946893 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:18.951611 (+ 4718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:18.953089 (+ 1478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:18.953096 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:18.953100 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:18.954105 (+ 1005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:18.954726 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:18.954733 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:18.954736 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:18.955208 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:18.955561 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:18.960372 (+ 4811us) server_negotiation.cc:299] Negotiation successful
0810 18:42:18.960786 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1318,"thread_start_us":1191,"threads_started":1}
I20260810 18:42:18.964679 16317 heartbeater.cc:347] Connected to a master server at 127.4.228.252:40375
I20260810 18:42:18.965251 16317 heartbeater.cc:464] Registering TS with master...
I20260810 18:42:18.967015 16317 heartbeater.cc:511] Master 127.4.228.252:40375 requested a full tablet report, sending...
I20260810 18:42:18.967422 16321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:18.926422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:39631 (local address 127.4.228.254:45361)
0810 18:42:18.926973 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:18.926980 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:18.929312 (+ 2332us) server_negotiation.cc:407] Connection header received
0810 18:42:18.933093 (+ 3781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:18.933100 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:18.933202 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:18.933348 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:18.935805 (+ 2457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:18.937107 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:18.939422 (+ 2315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:18.940173 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:18.940329 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:18.946531 (+ 6202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:18.946543 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:18.946552 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:18.946604 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:18.953099 (+ 6495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:18.960730 (+ 7631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:18.960742 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:18.960748 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:18.961022 (+ 274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:18.962189 (+ 1167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:18.962197 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:18.962203 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:18.963893 (+ 1690us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:18.965407 (+ 1514us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:18.966204 (+ 797us) server_negotiation.cc:299] Negotiation successful
0810 18:42:18.966690 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":1285,"threads_started":1}
I20260810 18:42:18.969069 16326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:18.927196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.4.228.195:39631)
0810 18:42:18.928543 (+ 1347us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:18.928616 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:18.931664 (+ 3048us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:18.933405 (+ 1741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:18.933413 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:18.934230 (+ 817us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:18.935606 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:18.935640 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:18.937292 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:18.937297 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:18.939242 (+ 1945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:18.939270 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:18.940448 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:18.940456 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:18.940805 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:18.941753 (+ 948us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:18.941807 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:18.946110 (+ 4303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:18.953317 (+ 7207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:18.953323 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:18.953327 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:18.953918 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:18.961168 (+ 7250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:18.961176 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:18.961183 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:18.961488 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:18.965325 (+ 3837us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:18.965332 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:18.965653 (+ 321us) client_negotiation.cc:769] Sending connection context
0810 18:42:18.965850 (+ 197us) client_negotiation.cc:240] Negotiation successful
0810 18:42:18.966246 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":964,"mutex_wait_us":320,"spinlock_wait_cycles":135808,"thread_start_us":626,"threads_started":1}
I20260810 18:42:18.969895 15567 ts_manager.cc:194] Registered new tserver with Master: b5068707091b42a5b99875e6389e158e (127.4.228.195:40897)
I20260810 18:42:18.971247 16318 heartbeater.cc:347] Connected to a master server at 127.4.228.254:45361
I20260810 18:42:18.971457 16318 heartbeater.cc:464] Registering TS with master...
I20260810 18:42:18.972187 16318 heartbeater.cc:511] Master 127.4.228.254:45361 requested a full tablet report, sending...
I20260810 18:42:18.973901 14334 ts_manager.cc:194] Re-registered known tserver with Master: b5068707091b42a5b99875e6389e158e (127.4.228.195:40897)
I20260810 18:42:18.975164 16325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:18.927227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:39135 (local address 127.4.228.195:34051)
0810 18:42:18.928543 (+ 1316us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:18.928616 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:18.931670 (+ 3054us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:18.932722 (+ 1052us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:18.932732 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:18.934230 (+ 1498us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:18.935606 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:18.935639 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:18.936831 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:18.936836 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:18.939886 (+ 3050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:18.939892 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:18.941730 (+ 1838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:18.941738 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:18.942234 (+ 496us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:18.942882 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:18.942907 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:18.957633 (+ 14726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:18.964954 (+ 7321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:18.964959 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:18.964964 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:18.965654 (+ 690us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:18.970939 (+ 5285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:18.970946 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:18.970950 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:18.972879 (+ 1929us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:18.973670 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:18.973676 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:18.973890 (+ 214us) client_negotiation.cc:769] Sending connection context
0810 18:42:18.974048 (+ 158us) client_negotiation.cc:240] Negotiation successful
0810 18:42:18.974416 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1095,"mutex_wait_us":1095,"spinlock_wait_cycles":30464,"thread_start_us":744,"threads_started":1}
I20260810 18:42:18.975478 16323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:18.926685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:34051 (local address 127.4.228.253:39135)
0810 18:42:18.927866 (+ 1181us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:18.927875 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:18.928759 (+ 884us) server_negotiation.cc:407] Connection header received
0810 18:42:18.932194 (+ 3435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:18.932200 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:18.932289 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:18.932404 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:18.935870 (+ 3466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:18.936646 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:18.940020 (+ 3374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:18.940625 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:18.940782 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:18.958378 (+ 17596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:18.958387 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:18.958393 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:18.958440 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:18.964698 (+ 6258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:18.970434 (+ 5736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:18.970440 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:18.970444 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:18.970769 (+ 325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:18.973027 (+ 2258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:18.973033 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:18.973037 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:18.973519 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:18.973782 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:18.974309 (+ 527us) server_negotiation.cc:299] Negotiation successful
0810 18:42:18.974685 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1029,"thread_start_us":879,"threads_started":1}
I20260810 18:42:18.976529 14334 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.228.195@KRBTEST.COM'} at 127.4.228.195:39631
I20260810 18:42:18.980882 16319 heartbeater.cc:347] Connected to a master server at 127.4.228.253:39135
I20260810 18:42:18.981155 16319 heartbeater.cc:464] Registering TS with master...
I20260810 18:42:18.981969 16319 heartbeater.cc:511] Master 127.4.228.253:39135 requested a full tablet report, sending...
I20260810 18:42:18.984977 15253 ts_manager.cc:194] Registered new tserver with Master: b5068707091b42a5b99875e6389e158e (127.4.228.195:40897)
I20260810 18:42:18.993957 16339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:18.946690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38806 (local address 127.4.228.254:45361)
0810 18:42:18.947631 (+ 941us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:18.947638 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:18.959707 (+ 12069us) server_negotiation.cc:407] Connection header received
0810 18:42:18.960343 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:18.960353 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:18.960472 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:18.960576 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:18.968673 (+ 8097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:18.970592 (+ 1919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:18.973731 (+ 3139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:18.974936 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:18.975084 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:18.982571 (+ 7487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:18.982578 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:18.982582 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:18.982648 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:18.988249 (+ 5601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:18.990455 (+ 2206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:18.990494 (+ 39us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:18.990497 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:18.990782 (+ 285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:18.991675 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:18.991682 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:18.991685 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:18.992243 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:18.992665 (+ 422us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:18.993112 (+ 447us) server_negotiation.cc:299] Negotiation successful
0810 18:42:18.993406 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":721,"thread_start_us":572,"threads_started":1}
I20260810 18:42:19.006722 16323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:18.953928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38936 (local address 127.4.228.253:39135)
0810 18:42:18.975701 (+ 21773us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:18.975705 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:18.980500 (+ 4795us) server_negotiation.cc:407] Connection header received
0810 18:42:18.980830 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:18.980837 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:18.981065 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:18.981157 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:18.985339 (+ 4182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:18.987067 (+ 1728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:18.989265 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:18.989947 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:18.990100 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:18.995366 (+ 5266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:18.995375 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:18.995381 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:18.995447 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:19.001294 (+ 5847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:19.002451 (+ 1157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:19.002493 (+ 42us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:19.002507 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:19.002758 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:19.003482 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:19.003490 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:19.003497 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:19.003922 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:19.004279 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:19.005313 (+ 1034us) server_negotiation.cc:299] Negotiation successful
0810 18:42:19.005734 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21679,"thread_start_us":25375,"threads_started":1}
I20260810 18:42:19.007257 16331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:18.981000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34654 (local address 127.4.228.252:40375)
0810 18:42:18.981210 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:18.981216 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:18.983922 (+ 2706us) server_negotiation.cc:407] Connection header received
0810 18:42:18.984779 (+ 857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:18.984785 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:18.984956 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:18.985794 (+ 838us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:18.988286 (+ 2492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:18.989381 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:18.991727 (+ 2346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:18.992520 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:18.992659 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:18.998137 (+ 5478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:18.998155 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:18.998160 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:18.998214 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:19.002731 (+ 4517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:19.003587 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:19.003604 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:19.003608 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:19.003837 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:19.004402 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:19.004407 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:19.004411 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:19.004960 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:19.005305 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:19.005690 (+ 385us) server_negotiation.cc:299] Negotiation successful
0810 18:42:19.006046 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 18:42:18.999996 14334 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38806:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:42:19.014021 14334 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:42:19.061261 16364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:19.047099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47134 (local address 127.4.228.194:44749)
0810 18:42:19.051337 (+ 4238us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:19.051346 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:19.052411 (+ 1065us) server_negotiation.cc:407] Connection header received
0810 18:42:19.052532 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:19.052539 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:19.052787 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:19.053084 (+ 297us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:42:19.053691 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:19.054947 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:19.057375 (+ 2428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:19.059776 (+ 2401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:19.059980 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:19.060299 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:19.060639 (+ 340us) server_negotiation.cc:299] Negotiation successful
0810 18:42:19.060871 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4015,"thread_start_us":3881,"threads_started":1}
I20260810 18:42:19.061342 16359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:19.046843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:44749 (local address 127.0.0.1:47134)
0810 18:42:19.051996 (+ 5153us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:19.052019 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:19.052246 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:19.053061 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:19.053066 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:19.053095 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:42:19.053514 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:19.053525 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:19.055100 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:19.055106 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:19.057063 (+ 1957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:19.057079 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:19.060156 (+ 3077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:19.060165 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:19.060423 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:19.060453 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 18:42:19.060571 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 18:42:19.060747 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5009,"thread_start_us":4729,"threads_started":1}
I20260810 18:42:19.072572 15983 tablet_service.cc:1467] Processing CreateTablet for tablet e9856f1cfdaa4f8da85ac78da4bdf282 (DEFAULT_TABLE table=test-table [id=6fbcc9951be645eba841ca2c755095a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:42:19.074460 16359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:19.052160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:40897 (local address 127.0.0.1:52782)
0810 18:42:19.061631 (+ 9471us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:19.061652 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:19.061762 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:19.065716 (+ 3954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:19.065732 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:19.065767 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:42:19.066062 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:19.066072 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:19.069103 (+ 3031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:19.069113 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:19.071534 (+ 2421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:19.071548 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:19.073493 (+ 1945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:19.073499 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:19.073724 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:19.073760 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 18:42:19.073869 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 18:42:19.074038 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9366,"thread_start_us":14707,"threads_started":1}
I20260810 18:42:19.074445 16366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:19.052887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52782 (local address 127.4.228.195:40897)
0810 18:42:19.064675 (+ 11788us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:19.064689 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:19.064769 (+ 80us) server_negotiation.cc:407] Connection header received
0810 18:42:19.064919 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:19.064933 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:19.065339 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:19.065719 (+ 380us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:42:19.066297 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:19.068434 (+ 2137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:19.071716 (+ 3282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:19.073286 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:19.073527 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:19.073784 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:19.073937 (+ 153us) server_negotiation.cc:299] Negotiation successful
0810 18:42:19.074105 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11158,"thread_start_us":11325,"threads_started":1}
I20260810 18:42:19.074718 15983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9856f1cfdaa4f8da85ac78da4bdf282. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:42:19.079463 16215 tablet_service.cc:1467] Processing CreateTablet for tablet e9856f1cfdaa4f8da85ac78da4bdf282 (DEFAULT_TABLE table=test-table [id=6fbcc9951be645eba841ca2c755095a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:42:19.081010 16371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:19.067651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36878 (local address 127.4.228.193:41347)
0810 18:42:19.071224 (+ 3573us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:19.071241 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:19.071285 (+ 44us) server_negotiation.cc:407] Connection header received
0810 18:42:19.071432 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:19.071441 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:19.071691 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:19.071967 (+ 276us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:42:19.072718 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:19.074110 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:19.077503 (+ 3393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:19.079502 (+ 1999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:19.079724 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:19.079951 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:19.080506 (+ 555us) server_negotiation.cc:299] Negotiation successful
0810 18:42:19.080686 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3363,"thread_start_us":3206,"threads_started":1}
I20260810 18:42:19.081032 16365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:19.067400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:41347 (local address 127.0.0.1:36878)
0810 18:42:19.067834 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:19.067854 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:19.068100 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:19.071945 (+ 3845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:19.071952 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:19.071993 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:42:19.072492 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:19.072506 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:19.074463 (+ 1957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:19.074469 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:19.077241 (+ 2772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:19.077261 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:19.079944 (+ 2683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:19.079953 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:19.080213 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:19.080254 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 18:42:19.080467 (+ 213us) client_negotiation.cc:240] Negotiation successful
0810 18:42:19.080642 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253}
I20260810 18:42:19.082088 16215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9856f1cfdaa4f8da85ac78da4bdf282. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:42:19.086079 15747 tablet_service.cc:1467] Processing CreateTablet for tablet e9856f1cfdaa4f8da85ac78da4bdf282 (DEFAULT_TABLE table=test-table [id=6fbcc9951be645eba841ca2c755095a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:42:19.087843 15747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9856f1cfdaa4f8da85ac78da4bdf282. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:42:19.091051 16375 tablet_bootstrap.cc:492] T e9856f1cfdaa4f8da85ac78da4bdf282 P 70f6632aa17f433ab1c0bf5502ea03a1: Bootstrap starting.
I20260810 18:42:19.094622 16375 tablet_bootstrap.cc:654] T e9856f1cfdaa4f8da85ac78da4bdf282 P 70f6632aa17f433ab1c0bf5502ea03a1: Neither blocks nor log segments found. Creating new log.
I20260810 18:42:19.095763 16375 log.cc:824] T e9856f1cfdaa4f8da85ac78da4bdf282 P 70f6632aa17f433ab1c0bf5502ea03a1: Log is configured to *not* fsync() on all Append() calls
I20260810 18:42:19.099809 16375 tablet_bootstrap.cc:492] T e9856f1cfdaa4f8da85ac78da4bdf282 P 70f6632aa17f433ab1c0bf5502ea03a1: No bootstrap required, opened a new log
I20260810 18:42:19.100153 16375 ts_tablet_manager.cc:1397] T e9856f1cfdaa4f8da85ac78da4bdf282 P 70f6632aa17f433ab1c0bf5502ea03a1: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260810 18:42:19.101156 16381 tablet_bootstrap.cc:492] T e9856f1cfdaa4f8da85ac78da4bdf282 P b5068707091b42a5b99875e6389e158e: Bootstrap starting.
I20260810 18:42:19.105247 16383 tablet_bootstrap.cc:492] T e9856f1cfdaa4f8da85ac78da4bdf282 P 6a9e3573e79244e681476d93147477ff: Bootstrap starting.
I20260810 18:42:19.106896 16381 tablet_bootstrap.cc:654] T e9856f1cfdaa4f8da85ac78da4bdf282 P b5068707091b42a5b99875e6389e158e: Neither blocks nor log segments found. Creating new log.
I20260810 18:42:19.109078 16381 log.cc:824] T e9856f1cfdaa4f8da85ac78da4bdf282 P b5068707091b42a5b99875e6389e158e: Log is configured to *not* fsync() on all Append() calls
I20260810 18:42:19.109788 16383 tablet_bootstrap.cc:654] T e9856f1cfdaa4f8da85ac78da4bdf282 P 6a9e3573e79244e681476d93147477ff: Neither blocks nor log segments found. Creating new log.
I20260810 18:42:19.112030 16383 log.cc:824] T e9856f1cfdaa4f8da85ac78da4bdf282 P 6a9e3573e79244e681476d93147477ff: Log is configured to *not* fsync() on all Append() calls
I20260810 18:42:19.113596 16375 raft_consensus.cc:348] T e9856f1cfdaa4f8da85ac78da4bdf282 P 70f6632aa17f433ab1c0bf5502ea03a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70f6632aa17f433ab1c0bf5502ea03a1" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44749 } } peers { permanent_uuid: "b5068707091b42a5b99875e6389e158e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40897 } } peers { permanent_uuid: "6a9e3573e79244e681476d93147477ff" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41347 } }
I20260810 18:42:19.113999 16381 tablet_bootstrap.cc:492] T e9856f1cfdaa4f8da85ac78da4bdf282 P b5068707091b42a5b99875e6389e158e: No bootstrap required, opened a new log
I20260810 18:42:19.114141 16375 raft_consensus.cc:374] T e9856f1cfdaa4f8da85ac78da4bdf282 P 70f6632aa17f433ab1c0bf5502ea03a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:42:19.114292 16375 raft_consensus.cc:729] T e9856f1cfdaa4f8da85ac78da4bdf282 P 70f6632aa17f433ab1c0bf5502ea03a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70f6632aa17f433ab1c0bf5502ea03a1, State: Initialized, Role: FOLLOWER
I20260810 18:42:19.114399 16381 ts_tablet_manager.cc:1397] T e9856f1cfdaa4f8da85ac78da4bdf282 P b5068707091b42a5b99875e6389e158e: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.004s
I20260810 18:42:19.115135 16375 consensus_queue.cc:260] T e9856f1cfdaa4f8da85ac78da4bdf282 P 70f6632aa17f433ab1c0bf5502ea03a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70f6632aa17f433ab1c0bf5502ea03a1" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44749 } } peers { permanent_uuid: "b5068707091b42a5b99875e6389e158e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40897 } } peers { permanent_uuid: "6a9e3573e79244e681476d93147477ff" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41347 } }
I20260810 18:42:19.116446 16383 tablet_bootstrap.cc:492] T e9856f1cfdaa4f8da85ac78da4bdf282 P 6a9e3573e79244e681476d93147477ff: No bootstrap required, opened a new log
I20260810 18:42:19.117050 16383 ts_tablet_manager.cc:1397] T e9856f1cfdaa4f8da85ac78da4bdf282 P 6a9e3573e79244e681476d93147477ff: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.004s
I20260810 18:42:19.117501 16375 ts_tablet_manager.cc:1428] T e9856f1cfdaa4f8da85ac78da4bdf282 P 70f6632aa17f433ab1c0bf5502ea03a1: Time spent starting tablet: real 0.017s user 0.017s sys 0.000s
I20260810 18:42:19.117825 16099 heartbeater.cc:503] Master 127.4.228.254:45361 was elected leader, sending a full tablet report...
I20260810 18:42:19.127169 16381 raft_consensus.cc:348] T e9856f1cfdaa4f8da85ac78da4bdf282 P b5068707091b42a5b99875e6389e158e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70f6632aa17f433ab1c0bf5502ea03a1" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44749 } } peers { permanent_uuid: "b5068707091b42a5b99875e6389e158e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40897 } } peers { permanent_uuid: "6a9e3573e79244e681476d93147477ff" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41347 } }
I20260810 18:42:19.127807 16381 raft_consensus.cc:374] T e9856f1cfdaa4f8da85ac78da4bdf282 P b5068707091b42a5b99875e6389e158e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:42:19.127931 16381 raft_consensus.cc:729] T e9856f1cfdaa4f8da85ac78da4bdf282 P b5068707091b42a5b99875e6389e158e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5068707091b42a5b99875e6389e158e, State: Initialized, Role: FOLLOWER
I20260810 18:42:19.128738 16381 consensus_queue.cc:260] T e9856f1cfdaa4f8da85ac78da4bdf282 P b5068707091b42a5b99875e6389e158e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70f6632aa17f433ab1c0bf5502ea03a1" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44749 } } peers { permanent_uuid: "b5068707091b42a5b99875e6389e158e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40897 } } peers { permanent_uuid: "6a9e3573e79244e681476d93147477ff" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41347 } }
I20260810 18:42:19.130290 16318 heartbeater.cc:503] Master 127.4.228.254:45361 was elected leader, sending a full tablet report...
I20260810 18:42:19.131268 16381 ts_tablet_manager.cc:1428] T e9856f1cfdaa4f8da85ac78da4bdf282 P b5068707091b42a5b99875e6389e158e: Time spent starting tablet: real 0.017s user 0.012s sys 0.004s
I20260810 18:42:19.133242 16383 raft_consensus.cc:348] T e9856f1cfdaa4f8da85ac78da4bdf282 P 6a9e3573e79244e681476d93147477ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70f6632aa17f433ab1c0bf5502ea03a1" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44749 } } peers { permanent_uuid: "b5068707091b42a5b99875e6389e158e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40897 } } peers { permanent_uuid: "6a9e3573e79244e681476d93147477ff" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41347 } }
I20260810 18:42:19.133837 16383 raft_consensus.cc:374] T e9856f1cfdaa4f8da85ac78da4bdf282 P 6a9e3573e79244e681476d93147477ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:42:19.133988 16383 raft_consensus.cc:729] T e9856f1cfdaa4f8da85ac78da4bdf282 P 6a9e3573e79244e681476d93147477ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a9e3573e79244e681476d93147477ff, State: Initialized, Role: FOLLOWER
I20260810 18:42:19.135466 16383 consensus_queue.cc:260] T e9856f1cfdaa4f8da85ac78da4bdf282 P 6a9e3573e79244e681476d93147477ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70f6632aa17f433ab1c0bf5502ea03a1" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44749 } } peers { permanent_uuid: "b5068707091b42a5b99875e6389e158e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40897 } } peers { permanent_uuid: "6a9e3573e79244e681476d93147477ff" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41347 } }
I20260810 18:42:19.140247 16383 ts_tablet_manager.cc:1428] T e9856f1cfdaa4f8da85ac78da4bdf282 P 6a9e3573e79244e681476d93147477ff: Time spent starting tablet: real 0.023s user 0.015s sys 0.004s
I20260810 18:42:19.142632 16388 raft_consensus.cc:482] T e9856f1cfdaa4f8da85ac78da4bdf282 P 70f6632aa17f433ab1c0bf5502ea03a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:42:19.143005 16388 raft_consensus.cc:504] T e9856f1cfdaa4f8da85ac78da4bdf282 P 70f6632aa17f433ab1c0bf5502ea03a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70f6632aa17f433ab1c0bf5502ea03a1" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44749 } } peers { permanent_uuid: "b5068707091b42a5b99875e6389e158e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40897 } } peers { permanent_uuid: "6a9e3573e79244e681476d93147477ff" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41347 } }
I20260810 18:42:19.145306 16388 leader_election.cc:288] T e9856f1cfdaa4f8da85ac78da4bdf282 P 70f6632aa17f433ab1c0bf5502ea03a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5068707091b42a5b99875e6389e158e (127.4.228.195:40897), 6a9e3573e79244e681476d93147477ff (127.4.228.193:41347)
I20260810 18:42:19.157285 16371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:19.146941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:43997 (local address 127.4.228.193:41347)
0810 18:42:19.147195 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:19.147201 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:19.147703 (+ 502us) server_negotiation.cc:407] Connection header received
0810 18:42:19.147855 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:19.147860 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:19.147941 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:19.148050 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:42:19.148908 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:19.150227 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:19.154300 (+ 4073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:19.155762 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:19.155983 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:19.156087 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:19.156823 (+ 736us) server_negotiation.cc:299] Negotiation successful
0810 18:42:19.156984 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260810 18:42:19.157274 16366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:19.145974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:34659 (local address 127.4.228.195:40897)
0810 18:42:19.146314 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:19.146321 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:19.146942 (+ 621us) server_negotiation.cc:407] Connection header received
0810 18:42:19.147435 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:19.147442 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:19.147579 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:19.147693 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:42:19.148906 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:19.150227 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:19.154300 (+ 4073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:19.155776 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:19.155983 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:19.156087 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:19.156823 (+ 736us) server_negotiation.cc:299] Negotiation successful
0810 18:42:19.156984 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260810 18:42:19.158020 16393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:19.146724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:41347 (local address 127.4.228.194:43997)
0810 18:42:19.147447 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:19.147465 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:19.147732 (+ 267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:19.148079 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:19.148086 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:19.148107 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:42:19.148714 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:19.148727 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:19.150508 (+ 1781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:19.150512 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:19.154032 (+ 3520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:19.154053 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:19.156255 (+ 2202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:19.156261 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:19.156517 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:19.156592 (+ 75us) client_negotiation.cc:769] Sending connection context
0810 18:42:19.156730 (+ 138us) client_negotiation.cc:240] Negotiation successful
0810 18:42:19.156956 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"mutex_wait_us":111,"thread_start_us":341,"threads_started":1}
I20260810 18:42:19.158154 16391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:19.145763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:40897 (local address 127.4.228.194:34659)
0810 18:42:19.146668 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:19.146711 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:19.147196 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:19.147932 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:19.147936 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:19.147956 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:42:19.148661 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:19.148681 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:19.150444 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:19.150450 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:19.154038 (+ 3588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:19.154053 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:19.156241 (+ 2188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:19.156253 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:19.156616 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:19.156650 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 18:42:19.156742 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 18:42:19.157013 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"mutex_wait_us":43,"thread_start_us":456,"threads_started":1}
I20260810 18:42:19.159612 16247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9856f1cfdaa4f8da85ac78da4bdf282" candidate_uuid: "70f6632aa17f433ab1c0bf5502ea03a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5068707091b42a5b99875e6389e158e" is_pre_election: true
I20260810 18:42:19.159515 15775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9856f1cfdaa4f8da85ac78da4bdf282" candidate_uuid: "70f6632aa17f433ab1c0bf5502ea03a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a9e3573e79244e681476d93147477ff" is_pre_election: true
I20260810 18:42:19.160203 16247 raft_consensus.cc:2454] T e9856f1cfdaa4f8da85ac78da4bdf282 P b5068707091b42a5b99875e6389e158e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70f6632aa17f433ab1c0bf5502ea03a1 in term 0.
I20260810 18:42:19.160224 15775 raft_consensus.cc:2454] T e9856f1cfdaa4f8da85ac78da4bdf282 P 6a9e3573e79244e681476d93147477ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70f6632aa17f433ab1c0bf5502ea03a1 in term 0.
I20260810 18:42:19.161067 15913 leader_election.cc:302] T e9856f1cfdaa4f8da85ac78da4bdf282 P 70f6632aa17f433ab1c0bf5502ea03a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a9e3573e79244e681476d93147477ff, 70f6632aa17f433ab1c0bf5502ea03a1; no voters:
I20260810 18:42:19.161548 16388 raft_consensus.cc:2781] T e9856f1cfdaa4f8da85ac78da4bdf282 P 70f6632aa17f433ab1c0bf5502ea03a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:42:19.161756 16388 raft_consensus.cc:482] T e9856f1cfdaa4f8da85ac78da4bdf282 P 70f6632aa17f433ab1c0bf5502ea03a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:42:19.161880 16388 raft_consensus.cc:3037] T e9856f1cfdaa4f8da85ac78da4bdf282 P 70f6632aa17f433ab1c0bf5502ea03a1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:42:19.164845 16388 raft_consensus.cc:504] T e9856f1cfdaa4f8da85ac78da4bdf282 P 70f6632aa17f433ab1c0bf5502ea03a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70f6632aa17f433ab1c0bf5502ea03a1" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44749 } } peers { permanent_uuid: "b5068707091b42a5b99875e6389e158e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40897 } } peers { permanent_uuid: "6a9e3573e79244e681476d93147477ff" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41347 } }
I20260810 18:42:19.165573 16388 leader_election.cc:288] T e9856f1cfdaa4f8da85ac78da4bdf282 P 70f6632aa17f433ab1c0bf5502ea03a1 [CANDIDATE]: Term 1 election: Requested vote from peers b5068707091b42a5b99875e6389e158e (127.4.228.195:40897), 6a9e3573e79244e681476d93147477ff (127.4.228.193:41347)
I20260810 18:42:19.166396 16247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9856f1cfdaa4f8da85ac78da4bdf282" candidate_uuid: "70f6632aa17f433ab1c0bf5502ea03a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5068707091b42a5b99875e6389e158e"
I20260810 18:42:19.166600 15775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9856f1cfdaa4f8da85ac78da4bdf282" candidate_uuid: "70f6632aa17f433ab1c0bf5502ea03a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a9e3573e79244e681476d93147477ff"
I20260810 18:42:19.166728 16247 raft_consensus.cc:3037] T e9856f1cfdaa4f8da85ac78da4bdf282 P b5068707091b42a5b99875e6389e158e [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:42:19.166893 15775 raft_consensus.cc:3037] T e9856f1cfdaa4f8da85ac78da4bdf282 P 6a9e3573e79244e681476d93147477ff [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:42:19.170413 16247 raft_consensus.cc:2454] T e9856f1cfdaa4f8da85ac78da4bdf282 P b5068707091b42a5b99875e6389e158e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70f6632aa17f433ab1c0bf5502ea03a1 in term 1.
I20260810 18:42:19.170423 15775 raft_consensus.cc:2454] T e9856f1cfdaa4f8da85ac78da4bdf282 P 6a9e3573e79244e681476d93147477ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70f6632aa17f433ab1c0bf5502ea03a1 in term 1.
I20260810 18:42:19.170965 15913 leader_election.cc:302] T e9856f1cfdaa4f8da85ac78da4bdf282 P 70f6632aa17f433ab1c0bf5502ea03a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70f6632aa17f433ab1c0bf5502ea03a1, b5068707091b42a5b99875e6389e158e; no voters:
I20260810 18:42:19.171427 16388 raft_consensus.cc:2781] T e9856f1cfdaa4f8da85ac78da4bdf282 P 70f6632aa17f433ab1c0bf5502ea03a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:42:19.172240 16388 raft_consensus.cc:686] T e9856f1cfdaa4f8da85ac78da4bdf282 P 70f6632aa17f433ab1c0bf5502ea03a1 [term 1 LEADER]: Becoming Leader. State: Replica: 70f6632aa17f433ab1c0bf5502ea03a1, State: Running, Role: LEADER
I20260810 18:42:19.173019 16388 consensus_queue.cc:237] T e9856f1cfdaa4f8da85ac78da4bdf282 P 70f6632aa17f433ab1c0bf5502ea03a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70f6632aa17f433ab1c0bf5502ea03a1" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44749 } } peers { permanent_uuid: "b5068707091b42a5b99875e6389e158e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40897 } } peers { permanent_uuid: "6a9e3573e79244e681476d93147477ff" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41347 } }
W20260810 18:42:19.176512 16320 tablet.cc:2369] T e9856f1cfdaa4f8da85ac78da4bdf282 P b5068707091b42a5b99875e6389e158e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:42:19.179726 14335 catalog_manager.cc:5495] T e9856f1cfdaa4f8da85ac78da4bdf282 P 70f6632aa17f433ab1c0bf5502ea03a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 70f6632aa17f433ab1c0bf5502ea03a1 (127.4.228.194). New cstate: current_term: 1 leader_uuid: "70f6632aa17f433ab1c0bf5502ea03a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70f6632aa17f433ab1c0bf5502ea03a1" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5068707091b42a5b99875e6389e158e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a9e3573e79244e681476d93147477ff" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41347 } health_report { overall_health: UNKNOWN } } }
W20260810 18:42:19.202913 15851 tablet.cc:2369] T e9856f1cfdaa4f8da85ac78da4bdf282 P 6a9e3573e79244e681476d93147477ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:42:19.220052 16364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:19.213221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47138 (local address 127.4.228.194:44749)
0810 18:42:19.213450 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:19.213455 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:19.213792 (+ 337us) server_negotiation.cc:407] Connection header received
0810 18:42:19.214044 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:19.214049 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:19.214146 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:19.214311 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:42:19.214891 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:19.215823 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:19.217283 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:19.217924 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:19.218045 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:19.218618 (+ 573us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:42:19.219223 (+ 605us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:42:19.219358 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:19.219552 (+ 194us) server_negotiation.cc:299] Negotiation successful
0810 18:42:19.219629 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260810 18:42:19.226341 16102 tablet.cc:2369] T e9856f1cfdaa4f8da85ac78da4bdf282 P 70f6632aa17f433ab1c0bf5502ea03a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:42:19.230631 16247 raft_consensus.cc:1261] T e9856f1cfdaa4f8da85ac78da4bdf282 P b5068707091b42a5b99875e6389e158e [term 1 FOLLOWER]: Refusing update from remote peer 70f6632aa17f433ab1c0bf5502ea03a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:42:19.230687 15775 raft_consensus.cc:1261] T e9856f1cfdaa4f8da85ac78da4bdf282 P 6a9e3573e79244e681476d93147477ff [term 1 FOLLOWER]: Refusing update from remote peer 70f6632aa17f433ab1c0bf5502ea03a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:42:19.231874 16388 consensus_queue.cc:1035] T e9856f1cfdaa4f8da85ac78da4bdf282 P 70f6632aa17f433ab1c0bf5502ea03a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a9e3573e79244e681476d93147477ff" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:42:19.232299 16394 consensus_queue.cc:1035] T e9856f1cfdaa4f8da85ac78da4bdf282 P 70f6632aa17f433ab1c0bf5502ea03a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5068707091b42a5b99875e6389e158e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:42:19.249114 16397 mvcc.cc:205] Tried to move back new op lower bound from 7317042541476450304 to 7317042541258469376. Current Snapshot: MvccSnapshot[applied={T|T < 7317042541476450304}]
I20260810 18:42:19.279484 14335 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:42:19.279968 14335 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:42:19.289072 14335 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c2d38638ade14167b99bd0ceb219791e: Sending DeleteTablet for 3 replicas of tablet e9856f1cfdaa4f8da85ac78da4bdf282
I20260810 18:42:19.291045 15983 tablet_service.cc:1514] Processing DeleteTablet for tablet e9856f1cfdaa4f8da85ac78da4bdf282 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:42:19 UTC) from {username='oryx', principal='oryx/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:47134
I20260810 18:42:19.291361 16215 tablet_service.cc:1514] Processing DeleteTablet for tablet e9856f1cfdaa4f8da85ac78da4bdf282 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:42:19 UTC) from {username='oryx', principal='oryx/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:52782
I20260810 18:42:19.293704 16415 tablet_replica.cc:330] T e9856f1cfdaa4f8da85ac78da4bdf282 P 70f6632aa17f433ab1c0bf5502ea03a1: stopping tablet replica
I20260810 18:42:19.294055 16416 tablet_replica.cc:330] T e9856f1cfdaa4f8da85ac78da4bdf282 P b5068707091b42a5b99875e6389e158e: stopping tablet replica
I20260810 18:42:19.294086 15747 tablet_service.cc:1514] Processing DeleteTablet for tablet e9856f1cfdaa4f8da85ac78da4bdf282 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:42:19 UTC) from {username='oryx', principal='oryx/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:36878
I20260810 18:42:19.294447 16415 raft_consensus.cc:2229] T e9856f1cfdaa4f8da85ac78da4bdf282 P 70f6632aa17f433ab1c0bf5502ea03a1 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:42:19.294606 16416 raft_consensus.cc:2229] T e9856f1cfdaa4f8da85ac78da4bdf282 P b5068707091b42a5b99875e6389e158e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:42:19.295636 16416 raft_consensus.cc:2258] T e9856f1cfdaa4f8da85ac78da4bdf282 P b5068707091b42a5b99875e6389e158e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:42:19.295972 16415 raft_consensus.cc:2258] T e9856f1cfdaa4f8da85ac78da4bdf282 P 70f6632aa17f433ab1c0bf5502ea03a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:42:19.301483 16416 ts_tablet_manager.cc:1905] T e9856f1cfdaa4f8da85ac78da4bdf282 P b5068707091b42a5b99875e6389e158e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:42:19.306772 16415 ts_tablet_manager.cc:1905] T e9856f1cfdaa4f8da85ac78da4bdf282 P 70f6632aa17f433ab1c0bf5502ea03a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:42:19.307245 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 14282
I20260810 18:42:19.308854 14378 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:42:19.311326 16417 tablet_replica.cc:330] T e9856f1cfdaa4f8da85ac78da4bdf282 P 6a9e3573e79244e681476d93147477ff: stopping tablet replica
I20260810 18:42:19.312043 16417 raft_consensus.cc:2229] T e9856f1cfdaa4f8da85ac78da4bdf282 P 6a9e3573e79244e681476d93147477ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:42:19.313021 16417 raft_consensus.cc:2258] T e9856f1cfdaa4f8da85ac78da4bdf282 P 6a9e3573e79244e681476d93147477ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:42:19.315841 16416 ts_tablet_manager.cc:1918] T e9856f1cfdaa4f8da85ac78da4bdf282 P b5068707091b42a5b99875e6389e158e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:42:19.316193 16416 log.cc:1196] T e9856f1cfdaa4f8da85ac78da4bdf282 P b5068707091b42a5b99875e6389e158e: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-2/wal/wals/e9856f1cfdaa4f8da85ac78da4bdf282
I20260810 18:42:19.316275 16417 ts_tablet_manager.cc:1905] T e9856f1cfdaa4f8da85ac78da4bdf282 P 6a9e3573e79244e681476d93147477ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:42:19.317188 16416 ts_tablet_manager.cc:1939] T e9856f1cfdaa4f8da85ac78da4bdf282 P b5068707091b42a5b99875e6389e158e: Deleting consensus metadata
I20260810 18:42:19.329550 16417 ts_tablet_manager.cc:1918] T e9856f1cfdaa4f8da85ac78da4bdf282 P 6a9e3573e79244e681476d93147477ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:42:19.329557 16415 ts_tablet_manager.cc:1918] T e9856f1cfdaa4f8da85ac78da4bdf282 P 70f6632aa17f433ab1c0bf5502ea03a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:42:19.329779 16417 log.cc:1196] T e9856f1cfdaa4f8da85ac78da4bdf282 P 6a9e3573e79244e681476d93147477ff: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-0/wal/wals/e9856f1cfdaa4f8da85ac78da4bdf282
I20260810 18:42:19.329818 16415 log.cc:1196] T e9856f1cfdaa4f8da85ac78da4bdf282 P 70f6632aa17f433ab1c0bf5502ea03a1: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-1/wal/wals/e9856f1cfdaa4f8da85ac78da4bdf282
I20260810 18:42:19.330591 16417 ts_tablet_manager.cc:1939] T e9856f1cfdaa4f8da85ac78da4bdf282 P 6a9e3573e79244e681476d93147477ff: Deleting consensus metadata
I20260810 18:42:19.330708 16415 ts_tablet_manager.cc:1939] T e9856f1cfdaa4f8da85ac78da4bdf282 P 70f6632aa17f433ab1c0bf5502ea03a1: Deleting consensus metadata
I20260810 18:42:19.337116 16323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:19.299527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38944 (local address 127.4.228.253:39135)
0810 18:42:19.299745 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:19.299752 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:19.307916 (+ 8164us) server_negotiation.cc:407] Connection header received
0810 18:42:19.308012 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:19.308018 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:19.308130 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:19.308269 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:19.314586 (+ 6317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:19.316580 (+ 1994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:19.320879 (+ 4299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:19.321838 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:19.322042 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:19.328206 (+ 6164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:19.328215 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:19.328221 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:19.328285 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:19.333220 (+ 4935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:19.334151 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:19.334157 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:19.334161 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:19.334338 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:19.334890 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:19.334896 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:19.334899 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:19.335378 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:19.335724 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:19.336111 (+ 387us) server_negotiation.cc:299] Negotiation successful
0810 18:42:19.336389 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 18:42:19.355698 16331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:19.305882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34662 (local address 127.4.228.252:40375)
0810 18:42:19.306134 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:19.306141 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:19.336748 (+ 30607us) server_negotiation.cc:407] Connection header received
0810 18:42:19.337020 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:19.337027 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:19.337120 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:19.337193 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:19.339817 (+ 2624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:19.340981 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:19.342858 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:19.343605 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:19.343767 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:19.347807 (+ 4040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:19.347816 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:19.347822 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:19.347874 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:19.352027 (+ 4153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:19.352989 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:19.352995 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:19.352999 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:19.353177 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:19.353654 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:19.353660 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:19.353663 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:19.354056 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:19.354383 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:19.354812 (+ 429us) server_negotiation.cc:299] Negotiation successful
0810 18:42:19.355068 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260810 18:42:19.464098 14311 catalog_manager.cc:4841] TS b5068707091b42a5b99875e6389e158e (127.4.228.195:40897): tablet e9856f1cfdaa4f8da85ac78da4bdf282 (table test-table [id=6fbcc9951be645eba841ca2c755095a3]) successfully deleted
I20260810 18:42:19.464624 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 14282
W20260810 18:42:19.498847 15669 connection.cc:535] server connection from 127.0.0.1:36878 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 18:42:19.500257 15910 connection.cc:535] server connection from 127.0.0.1:47134 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 18:42:19.523306 16323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:19.504087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38958 (local address 127.4.228.253:39135)
0810 18:42:19.504355 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:19.504365 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:19.505635 (+ 1270us) server_negotiation.cc:407] Connection header received
0810 18:42:19.505983 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:19.505989 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:19.506083 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:19.506203 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:19.509903 (+ 3700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:19.511019 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:19.512464 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:19.513256 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:19.513402 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:19.516463 (+ 3061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:19.516472 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:19.516479 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:19.516529 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:19.519690 (+ 3161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:19.520821 (+ 1131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:19.520828 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:19.520832 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:19.521034 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:19.521463 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:19.521468 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:19.521477 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:19.521801 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:19.522079 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:19.522437 (+ 358us) server_negotiation.cc:299] Negotiation successful
0810 18:42:19.522729 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260810 18:42:19.525245 16331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:19.505899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34668 (local address 127.4.228.252:40375)
0810 18:42:19.506172 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:19.506180 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:19.506199 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:42:19.506258 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:19.506263 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:19.506340 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:19.506434 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:19.508448 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:19.509222 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:19.510968 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:19.511746 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:19.511907 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:19.515608 (+ 3701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:19.515616 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:19.515622 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:19.515672 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:19.519689 (+ 4017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:19.521218 (+ 1529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:19.521224 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:19.521228 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:19.521397 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:19.522557 (+ 1160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:19.522563 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:19.522566 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:19.522980 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:19.523255 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:19.524814 (+ 1559us) server_negotiation.cc:299] Negotiation successful
0810 18:42:19.525015 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260810 18:42:20.261175 16430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:20.260073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.4.228.193:40087)
0810 18:42:20.260718 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:20.260867 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":352,"threads_started":1}
W20260810 18:42:20.261888 15849 heartbeater.cc:644] Failed to heartbeat to 127.4.228.254:45361 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
I20260810 18:42:20.265317 16431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:20.264164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.4.228.194:34507)
0810 18:42:20.264872 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:20.265040 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":407,"threads_started":1}
W20260810 18:42:20.265991 16099 heartbeater.cc:644] Failed to heartbeat to 127.4.228.254:45361 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
I20260810 18:42:20.266978 16432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:20.264969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.4.228.195:56009)
0810 18:42:20.266609 (+ 1640us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:20.266758 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1464,"thread_start_us":1536,"threads_started":1}
W20260810 18:42:20.267484 16318 heartbeater.cc:644] Failed to heartbeat to 127.4.228.254:45361 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
I20260810 18:42:20.790767 16434 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c2d38638ade14167b99bd0ceb219791e)
I20260810 18:42:20.790917 16435 raft_consensus.cc:482] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c2d38638ade14167b99bd0ceb219791e)
I20260810 18:42:20.791137 16434 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: false } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: false } }
I20260810 18:42:20.791159 16435 raft_consensus.cc:504] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: false } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: false } }
I20260810 18:42:20.792950 16435 leader_election.cc:288] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c2d38638ade14167b99bd0ceb219791e (127.4.228.254:45361), fd3cfc2801be46d4b2c030180a0d9a72 (127.4.228.252:40375)
I20260810 18:42:20.792958 16434 leader_election.cc:288] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c2d38638ade14167b99bd0ceb219791e (127.4.228.254:45361), 882afacf744b4b6b89526c8f0108387e (127.4.228.253:39135)
I20260810 18:42:20.804555 16440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:20.793162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.0.0.1:39256)
0810 18:42:20.803811 (+ 10649us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:20.804150 (+ 339us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10432,"threads_started":1}
I20260810 18:42:20.806813 16439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:20.793123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.0.0.1:39254)
0810 18:42:20.806088 (+ 12965us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:20.806396 (+ 308us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12783,"threads_started":1}
W20260810 18:42:20.808671 15539 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
W20260810 18:42:20.809862 15233 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
I20260810 18:42:20.816118 16440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:20.793451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:40375 (local address 127.0.0.1:34682)
0810 18:42:20.805228 (+ 11777us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:20.805255 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:20.805472 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:20.809121 (+ 3649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:20.809126 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:20.809158 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:42:20.809515 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:20.809533 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:20.811396 (+ 1863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:20.811402 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:20.813532 (+ 2130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:20.813547 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:20.815294 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:20.815300 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:20.815474 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:20.815524 (+ 50us) client_negotiation.cc:769] Sending connection context
0810 18:42:20.815607 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 18:42:20.815782 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11647,"thread_start_us":11357,"threads_started":1}
I20260810 18:42:20.816615 16439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:20.793398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:39135 (local address 127.0.0.1:38970)
0810 18:42:20.807231 (+ 13833us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:20.807265 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:20.807477 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:20.808549 (+ 1072us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:20.808555 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:20.808590 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:42:20.808903 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:20.808918 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:20.811397 (+ 2479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:20.811402 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:20.813514 (+ 2112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:20.813533 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:20.815760 (+ 2227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:20.815766 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:20.815961 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:20.815995 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 18:42:20.816069 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 18:42:20.816261 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13702,"thread_start_us":12355,"threads_started":1}
I20260810 18:42:20.817566 16442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:20.797509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34682 (local address 127.4.228.252:40375)
0810 18:42:20.808652 (+ 11143us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:20.808664 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:20.808682 (+ 18us) server_negotiation.cc:407] Connection header received
0810 18:42:20.808731 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:20.808735 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:20.808819 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:20.808929 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:42:20.809763 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:20.811233 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:20.813765 (+ 2532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:20.815101 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:20.816901 (+ 1800us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:20.817023 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:20.817079 (+ 56us) server_negotiation.cc:299] Negotiation successful
0810 18:42:20.817210 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10993,"thread_start_us":10841,"threads_started":1}
I20260810 18:42:20.817626 16441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:20.793703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38970 (local address 127.4.228.253:39135)
0810 18:42:20.808145 (+ 14442us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:20.808153 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:20.808179 (+ 26us) server_negotiation.cc:407] Connection header received
0810 18:42:20.808257 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:20.808266 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:20.808361 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:20.809555 (+ 1194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:42:20.809815 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:20.811233 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:20.813797 (+ 2564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:20.815555 (+ 1758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:20.816977 (+ 1422us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:20.817096 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:20.817152 (+ 56us) server_negotiation.cc:299] Negotiation successful
0810 18:42:20.817274 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14281,"thread_start_us":14072,"threads_started":1}
I20260810 18:42:20.818331 15274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "882afacf744b4b6b89526c8f0108387e" is_pre_election: true
I20260810 18:42:20.818830 15274 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd3cfc2801be46d4b2c030180a0d9a72 in term 1.
I20260810 18:42:20.819181 15584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "882afacf744b4b6b89526c8f0108387e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" is_pre_election: true
I20260810 18:42:20.819597 15584 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 882afacf744b4b6b89526c8f0108387e in term 1.
I20260810 18:42:20.819664 16440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:20.819220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.0.0.1:39290)
0810 18:42:20.819363 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:20.819457 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 18:42:20.819626 15540 leader_election.cc:302] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 882afacf744b4b6b89526c8f0108387e, fd3cfc2801be46d4b2c030180a0d9a72; no voters:
W20260810 18:42:20.819967 15233 leader_election.cc:334] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c2d38638ade14167b99bd0ceb219791e (127.4.228.254:45361): Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
I20260810 18:42:20.820086 16434 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 18:42:20.820214 16439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:20.819807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.0.0.1:39292)
0810 18:42:20.819952 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:20.820038 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260810 18:42:20.820242 16434 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c2d38638ade14167b99bd0ceb219791e)
I20260810 18:42:20.820346 16434 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [term 1 FOLLOWER]: Advancing to term 2
W20260810 18:42:20.820492 15539 leader_election.cc:334] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c2d38638ade14167b99bd0ceb219791e (127.4.228.254:45361): Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
I20260810 18:42:20.820806 15234 leader_election.cc:302] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 882afacf744b4b6b89526c8f0108387e, fd3cfc2801be46d4b2c030180a0d9a72; no voters: c2d38638ade14167b99bd0ceb219791e
I20260810 18:42:20.821237 16435 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 18:42:20.821409 16435 raft_consensus.cc:482] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [term 1 FOLLOWER]: Starting leader election (detected failure of leader c2d38638ade14167b99bd0ceb219791e)
I20260810 18:42:20.821543 16435 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [term 1 FOLLOWER]: Advancing to term 2
I20260810 18:42:20.825078 16434 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: false } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: false } }
I20260810 18:42:20.825178 16435 raft_consensus.cc:504] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: false } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: false } }
I20260810 18:42:20.825875 16434 leader_election.cc:288] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [CANDIDATE]: Term 2 election: Requested vote from peers c2d38638ade14167b99bd0ceb219791e (127.4.228.254:45361), 882afacf744b4b6b89526c8f0108387e (127.4.228.253:39135)
I20260810 18:42:20.825979 16435 leader_election.cc:288] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [CANDIDATE]: Term 2 election: Requested vote from peers c2d38638ade14167b99bd0ceb219791e (127.4.228.254:45361), fd3cfc2801be46d4b2c030180a0d9a72 (127.4.228.252:40375)
I20260810 18:42:20.826381 16439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:20.825969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.0.0.1:39302)
0810 18:42:20.826126 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:20.826229 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260810 18:42:20.826565 15274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "882afacf744b4b6b89526c8f0108387e"
I20260810 18:42:20.826756 16440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:20.826278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.0.0.1:39318)
0810 18:42:20.826432 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:20.826595 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 18:42:20.826925 15274 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate fd3cfc2801be46d4b2c030180a0d9a72 in current term 2: Already voted for candidate 882afacf744b4b6b89526c8f0108387e in this term.
I20260810 18:42:20.827382 15584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "882afacf744b4b6b89526c8f0108387e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fd3cfc2801be46d4b2c030180a0d9a72"
I20260810 18:42:20.827710 15584 raft_consensus.cc:2379] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 882afacf744b4b6b89526c8f0108387e in current term 2: Already voted for candidate fd3cfc2801be46d4b2c030180a0d9a72 in this term.
I20260810 18:42:20.827849 16439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:20.827404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.0.0.1:39320)
0810 18:42:20.827574 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:20.827659 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260810 18:42:20.828212 15539 leader_election.cc:334] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c2d38638ade14167b99bd0ceb219791e (127.4.228.254:45361): Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
I20260810 18:42:20.828372 15539 leader_election.cc:302] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fd3cfc2801be46d4b2c030180a0d9a72; no voters: 882afacf744b4b6b89526c8f0108387e, c2d38638ade14167b99bd0ceb219791e
I20260810 18:42:20.828529 16440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:20.828024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.0.0.1:39326)
0810 18:42:20.828244 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:20.828364 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260810 18:42:20.828713 16434 raft_consensus.cc:2726] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260810 18:42:20.828754 15233 leader_election.cc:334] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c2d38638ade14167b99bd0ceb219791e (127.4.228.254:45361): Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
I20260810 18:42:20.828907 15233 leader_election.cc:302] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 882afacf744b4b6b89526c8f0108387e; no voters: c2d38638ade14167b99bd0ceb219791e, fd3cfc2801be46d4b2c030180a0d9a72
I20260810 18:42:20.829228 16435 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260810 18:42:21.263967 16446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:21.262910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.4.228.193:42937)
0810 18:42:21.263596 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:21.263715 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":344,"threads_started":1}
I20260810 18:42:21.268863 16449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:21.268087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.4.228.195:34591)
0810 18:42:21.268559 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:21.268638 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":256,"threads_started":1}
I20260810 18:42:22.266232 16450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:22.265335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.4.228.193:41757)
0810 18:42:22.265900 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:22.266002 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":257,"threads_started":1}
I20260810 18:42:22.270617 16453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:22.269709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.4.228.195:38031)
0810 18:42:22.270264 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:22.270355 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":320,"threads_started":1}
I20260810 18:42:22.275049 16447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:21.266697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.4.228.194:57415)
0810 18:42:21.267417 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:22.274806 (+1007389us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":509,"threads_started":1}
I20260810 18:42:22.601017 16454 raft_consensus.cc:482] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:42:22.601274 16454 raft_consensus.cc:504] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: false } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: false } }
I20260810 18:42:22.602087 16454 leader_election.cc:288] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c2d38638ade14167b99bd0ceb219791e (127.4.228.254:45361), fd3cfc2801be46d4b2c030180a0d9a72 (127.4.228.252:40375)
I20260810 18:42:22.602830 15584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "882afacf744b4b6b89526c8f0108387e" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" is_pre_election: true
I20260810 18:42:22.603157 15584 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 882afacf744b4b6b89526c8f0108387e in term 2.
I20260810 18:42:22.603225 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:22.602170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.0.0.1:39574)
0810 18:42:22.602840 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:22.602965 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":386,"threads_started":1}
I20260810 18:42:22.603801 15234 leader_election.cc:302] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 882afacf744b4b6b89526c8f0108387e, fd3cfc2801be46d4b2c030180a0d9a72; no voters:
I20260810 18:42:22.604203 16454 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260810 18:42:22.604370 16454 raft_consensus.cc:482] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:42:22.604525 16454 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [term 2 FOLLOWER]: Advancing to term 3
I20260810 18:42:22.605036 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:22.604659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.0.0.1:39588)
0810 18:42:22.604817 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:22.604906 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 18:42:22.605224 15233 leader_election.cc:334] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c2d38638ade14167b99bd0ceb219791e (127.4.228.254:45361): Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
I20260810 18:42:22.607318 16454 raft_consensus.cc:504] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: false } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: false } }
I20260810 18:42:22.608022 16454 leader_election.cc:288] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [CANDIDATE]: Term 3 election: Requested vote from peers c2d38638ade14167b99bd0ceb219791e (127.4.228.254:45361), fd3cfc2801be46d4b2c030180a0d9a72 (127.4.228.252:40375)
I20260810 18:42:22.608788 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:22.608237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.0.0.1:39596)
0810 18:42:22.608429 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:22.608588 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 18:42:22.608733 15584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "882afacf744b4b6b89526c8f0108387e" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fd3cfc2801be46d4b2c030180a0d9a72"
I20260810 18:42:22.609019 15584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [term 2 FOLLOWER]: Advancing to term 3
I20260810 18:42:22.609972 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:22.609511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.0.0.1:39606)
0810 18:42:22.609699 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:22.609807 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260810 18:42:22.610327 15233 leader_election.cc:334] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer c2d38638ade14167b99bd0ceb219791e (127.4.228.254:45361): Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
I20260810 18:42:22.611825 15584 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 882afacf744b4b6b89526c8f0108387e in term 3.
I20260810 18:42:22.612356 15234 leader_election.cc:302] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 882afacf744b4b6b89526c8f0108387e, fd3cfc2801be46d4b2c030180a0d9a72; no voters: c2d38638ade14167b99bd0ceb219791e
I20260810 18:42:22.612692 16454 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [term 3 FOLLOWER]: Leader election won for term 3
I20260810 18:42:22.613178 16454 raft_consensus.cc:686] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [term 3 LEADER]: Becoming Leader. State: Replica: 882afacf744b4b6b89526c8f0108387e, State: Running, Role: LEADER
I20260810 18:42:22.613595 16454 consensus_queue.cc:237] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: false } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: false } }
I20260810 18:42:22.615945 16457 sys_catalog.cc:455] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 882afacf744b4b6b89526c8f0108387e. Latest consensus state: current_term: 3 leader_uuid: "882afacf744b4b6b89526c8f0108387e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: false } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: false } } }
I20260810 18:42:22.616384 16457 sys_catalog.cc:458] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [sys.catalog]: This master's current role is: LEADER
I20260810 18:42:22.617147 16460 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:42:22.621554 16460 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:42:22.622514 16460 catalog_manager.cc:1259] Loaded cluster ID: df523c3658514fd88ae4a2bfce9ed027
I20260810 18:42:22.622632 16460 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:42:22.623831 16460 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:42:22.624681 16460 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e: Loaded TSK: 0
I20260810 18:42:22.625694 16460 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:42:22.667186 15253 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38958:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:42:22.675076 15253 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:42:22.680174 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:22.679620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.0.0.1:39624)
0810 18:42:22.679853 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:22.679965 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260810 18:42:22.680303 15584 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [term 3 FOLLOWER]: Refusing update from remote peer 882afacf744b4b6b89526c8f0108387e: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260810 18:42:22.681360 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:22.680957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.0.0.1:39632)
0810 18:42:22.681159 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:22.681245 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 18:42:22.681617 15233 consensus_peers.cc:487] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e -> Peer c2d38638ade14167b99bd0ceb219791e (127.4.228.254:45361): Couldn't send request to peer c2d38638ade14167b99bd0ceb219791e. Status: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 18:42:22.681646 16457 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260810 18:42:22.685015 16462 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 882afacf744b4b6b89526c8f0108387e. Latest consensus state: current_term: 3 leader_uuid: "882afacf744b4b6b89526c8f0108387e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: false } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: false } } }
I20260810 18:42:22.685338 16462 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:42:22.686739 16454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "882afacf744b4b6b89526c8f0108387e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: false } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: false } } }
I20260810 18:42:22.687146 16454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e [sys.catalog]: This master's current role is: LEADER
I20260810 18:42:22.688038 16462 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "882afacf744b4b6b89526c8f0108387e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c2d38638ade14167b99bd0ceb219791e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45361 } } peers { permanent_uuid: "882afacf744b4b6b89526c8f0108387e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 39135 } attrs { promote: false } } peers { permanent_uuid: "fd3cfc2801be46d4b2c030180a0d9a72" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 40375 } attrs { promote: false } } }
I20260810 18:42:22.690639 16462 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd3cfc2801be46d4b2c030180a0d9a72 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:42:22.707389 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:22.696107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:44749 (local address 127.0.0.1:47140)
0810 18:42:22.696344 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:22.696367 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:22.696493 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:22.699589 (+ 3096us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:22.699595 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:22.699623 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:42:22.700026 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:22.700042 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:22.702718 (+ 2676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:22.702724 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:22.704799 (+ 2075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:22.704813 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:22.706525 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:22.706532 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:22.706715 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:22.706760 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 18:42:22.706835 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 18:42:22.706978 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":3}
I20260810 18:42:22.708896 16477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:22.696385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47140 (local address 127.4.228.194:44749)
0810 18:42:22.699056 (+ 2671us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:22.699063 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:22.699084 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:42:22.699224 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:22.699229 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:22.699339 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:22.699972 (+ 633us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:42:22.700281 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:22.702528 (+ 2247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:22.705018 (+ 2490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:22.706302 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:22.708223 (+ 1921us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:22.708338 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:22.708397 (+ 59us) server_negotiation.cc:299] Negotiation successful
0810 18:42:22.708542 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2536,"thread_start_us":2423,"threads_started":1}
I20260810 18:42:22.710178 16481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:22.696883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52790 (local address 127.4.228.195:40897)
0810 18:42:22.698622 (+ 1739us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:22.698632 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:22.699168 (+ 536us) server_negotiation.cc:407] Connection header received
0810 18:42:22.699248 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:22.699253 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:22.699335 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:22.699459 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:42:22.700977 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:22.702241 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:22.706272 (+ 4031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:22.707592 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:22.707717 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:22.707837 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:22.709692 (+ 1855us) server_negotiation.cc:299] Negotiation successful
0810 18:42:22.709847 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1565,"thread_start_us":1460,"threads_started":1}
I20260810 18:42:22.710225 16478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:22.696580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:40897 (local address 127.0.0.1:52790)
0810 18:42:22.698888 (+ 2308us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:22.698908 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:22.699039 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:22.699732 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:22.699736 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:22.699756 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:42:22.700734 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:22.700747 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:22.702410 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:22.702415 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:22.706058 (+ 3643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:22.706077 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:22.708032 (+ 1955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:22.708037 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:22.708226 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:22.708260 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 18:42:22.709662 (+ 1402us) client_negotiation.cc:240] Negotiation successful
0810 18:42:22.709805 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2177,"mutex_wait_us":112,"thread_start_us":2977,"threads_started":1}
I20260810 18:42:22.711696 15983 tablet_service.cc:1467] Processing CreateTablet for tablet 44e2c76ae23f4724bc313aa1e51e5316 (DEFAULT_TABLE table=test-table [id=5adbf3346d6746fea0cb2a0d0fa78763]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:42:22.711824 16215 tablet_service.cc:1467] Processing CreateTablet for tablet 44e2c76ae23f4724bc313aa1e51e5316 (DEFAULT_TABLE table=test-table [id=5adbf3346d6746fea0cb2a0d0fa78763]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:42:22.712246 15983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44e2c76ae23f4724bc313aa1e51e5316. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:42:22.712455 16215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44e2c76ae23f4724bc313aa1e51e5316. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:42:22.718077 16484 tablet_bootstrap.cc:492] T 44e2c76ae23f4724bc313aa1e51e5316 P 70f6632aa17f433ab1c0bf5502ea03a1: Bootstrap starting.
I20260810 18:42:22.719862 16484 tablet_bootstrap.cc:654] T 44e2c76ae23f4724bc313aa1e51e5316 P 70f6632aa17f433ab1c0bf5502ea03a1: Neither blocks nor log segments found. Creating new log.
I20260810 18:42:22.722999 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:22.700041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:41347 (local address 127.0.0.1:36890)
0810 18:42:22.707643 (+ 7602us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:22.707665 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:22.707763 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:22.716059 (+ 8296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:22.716066 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:22.716097 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:42:22.716273 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:22.716284 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:22.717761 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:22.717768 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:22.719735 (+ 1967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:22.719754 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:22.721772 (+ 2018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:22.721783 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:22.722080 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:22.722189 (+ 109us) client_negotiation.cc:769] Sending connection context
0810 18:42:22.722319 (+ 130us) client_negotiation.cc:240] Negotiation successful
0810 18:42:22.722551 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7491,"thread_start_us":10652,"threads_started":1}
I20260810 18:42:22.723105 16483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:22.700143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36890 (local address 127.4.228.193:41347)
0810 18:42:22.715665 (+ 15522us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:22.715671 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:22.715693 (+ 22us) server_negotiation.cc:407] Connection header received
0810 18:42:22.715782 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:22.715787 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:22.715873 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:22.715987 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:42:22.716450 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:22.717566 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:22.719956 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:22.721551 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:22.721743 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:22.721912 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:22.722379 (+ 467us) server_negotiation.cc:299] Negotiation successful
0810 18:42:22.722615 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15350,"thread_start_us":15969,"threads_started":1}
I20260810 18:42:22.724303 16485 tablet_bootstrap.cc:492] T 44e2c76ae23f4724bc313aa1e51e5316 P b5068707091b42a5b99875e6389e158e: Bootstrap starting.
I20260810 18:42:22.724474 15747 tablet_service.cc:1467] Processing CreateTablet for tablet 44e2c76ae23f4724bc313aa1e51e5316 (DEFAULT_TABLE table=test-table [id=5adbf3346d6746fea0cb2a0d0fa78763]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:42:22.724956 15747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44e2c76ae23f4724bc313aa1e51e5316. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:42:22.726146 16485 tablet_bootstrap.cc:654] T 44e2c76ae23f4724bc313aa1e51e5316 P b5068707091b42a5b99875e6389e158e: Neither blocks nor log segments found. Creating new log.
I20260810 18:42:22.729353 16484 tablet_bootstrap.cc:492] T 44e2c76ae23f4724bc313aa1e51e5316 P 70f6632aa17f433ab1c0bf5502ea03a1: No bootstrap required, opened a new log
I20260810 18:42:22.729372 16485 tablet_bootstrap.cc:492] T 44e2c76ae23f4724bc313aa1e51e5316 P b5068707091b42a5b99875e6389e158e: No bootstrap required, opened a new log
I20260810 18:42:22.729529 16484 ts_tablet_manager.cc:1397] T 44e2c76ae23f4724bc313aa1e51e5316 P 70f6632aa17f433ab1c0bf5502ea03a1: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.004s
I20260810 18:42:22.729557 16485 ts_tablet_manager.cc:1397] T 44e2c76ae23f4724bc313aa1e51e5316 P b5068707091b42a5b99875e6389e158e: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 18:42:22.730250 16485 raft_consensus.cc:348] T 44e2c76ae23f4724bc313aa1e51e5316 P b5068707091b42a5b99875e6389e158e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70f6632aa17f433ab1c0bf5502ea03a1" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44749 } } peers { permanent_uuid: "b5068707091b42a5b99875e6389e158e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40897 } } peers { permanent_uuid: "6a9e3573e79244e681476d93147477ff" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41347 } }
I20260810 18:42:22.730461 16485 raft_consensus.cc:374] T 44e2c76ae23f4724bc313aa1e51e5316 P b5068707091b42a5b99875e6389e158e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:42:22.730388 16484 raft_consensus.cc:348] T 44e2c76ae23f4724bc313aa1e51e5316 P 70f6632aa17f433ab1c0bf5502ea03a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70f6632aa17f433ab1c0bf5502ea03a1" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44749 } } peers { permanent_uuid: "b5068707091b42a5b99875e6389e158e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40897 } } peers { permanent_uuid: "6a9e3573e79244e681476d93147477ff" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41347 } }
I20260810 18:42:22.730610 16485 raft_consensus.cc:729] T 44e2c76ae23f4724bc313aa1e51e5316 P b5068707091b42a5b99875e6389e158e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5068707091b42a5b99875e6389e158e, State: Initialized, Role: FOLLOWER
I20260810 18:42:22.730690 16484 raft_consensus.cc:374] T 44e2c76ae23f4724bc313aa1e51e5316 P 70f6632aa17f433ab1c0bf5502ea03a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:42:22.730844 16484 raft_consensus.cc:729] T 44e2c76ae23f4724bc313aa1e51e5316 P 70f6632aa17f433ab1c0bf5502ea03a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70f6632aa17f433ab1c0bf5502ea03a1, State: Initialized, Role: FOLLOWER
I20260810 18:42:22.730908 16485 consensus_queue.cc:260] T 44e2c76ae23f4724bc313aa1e51e5316 P b5068707091b42a5b99875e6389e158e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70f6632aa17f433ab1c0bf5502ea03a1" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44749 } } peers { permanent_uuid: "b5068707091b42a5b99875e6389e158e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40897 } } peers { permanent_uuid: "6a9e3573e79244e681476d93147477ff" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41347 } }
I20260810 18:42:22.731109 16484 consensus_queue.cc:260] T 44e2c76ae23f4724bc313aa1e51e5316 P 70f6632aa17f433ab1c0bf5502ea03a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70f6632aa17f433ab1c0bf5502ea03a1" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44749 } } peers { permanent_uuid: "b5068707091b42a5b99875e6389e158e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40897 } } peers { permanent_uuid: "6a9e3573e79244e681476d93147477ff" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41347 } }
I20260810 18:42:22.735178 16493 tablet_bootstrap.cc:492] T 44e2c76ae23f4724bc313aa1e51e5316 P 6a9e3573e79244e681476d93147477ff: Bootstrap starting.
I20260810 18:42:22.737922 16493 tablet_bootstrap.cc:654] T 44e2c76ae23f4724bc313aa1e51e5316 P 6a9e3573e79244e681476d93147477ff: Neither blocks nor log segments found. Creating new log.
I20260810 18:42:22.738934 16484 ts_tablet_manager.cc:1428] T 44e2c76ae23f4724bc313aa1e51e5316 P 70f6632aa17f433ab1c0bf5502ea03a1: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260810 18:42:22.740193 16447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:22.739742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.4.228.194:37439)
0810 18:42:22.739924 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:22.740022 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 18:42:22.741822 16447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:22.741326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.4.228.194:46553)
0810 18:42:22.741561 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:22.741656 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260810 18:42:22.743618 16485 ts_tablet_manager.cc:1428] T 44e2c76ae23f4724bc313aa1e51e5316 P b5068707091b42a5b99875e6389e158e: Time spent starting tablet: real 0.014s user 0.003s sys 0.010s
I20260810 18:42:22.745122 16453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:22.744676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.4.228.195:59767)
0810 18:42:22.744868 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:22.744970 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260810 18:42:22.746304 16100 heartbeater.cc:503] Master 127.4.228.253:39135 was elected leader, sending a full tablet report...
I20260810 18:42:22.746448 16453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:22.745935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.4.228.195:40715)
0810 18:42:22.746178 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:22.746266 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260810 18:42:22.746994 16319 heartbeater.cc:503] Master 127.4.228.253:39135 was elected leader, sending a full tablet report...
I20260810 18:42:22.748603 16493 tablet_bootstrap.cc:492] T 44e2c76ae23f4724bc313aa1e51e5316 P 6a9e3573e79244e681476d93147477ff: No bootstrap required, opened a new log
I20260810 18:42:22.748797 16493 ts_tablet_manager.cc:1397] T 44e2c76ae23f4724bc313aa1e51e5316 P 6a9e3573e79244e681476d93147477ff: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.005s
I20260810 18:42:22.749609 16493 raft_consensus.cc:348] T 44e2c76ae23f4724bc313aa1e51e5316 P 6a9e3573e79244e681476d93147477ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70f6632aa17f433ab1c0bf5502ea03a1" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44749 } } peers { permanent_uuid: "b5068707091b42a5b99875e6389e158e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40897 } } peers { permanent_uuid: "6a9e3573e79244e681476d93147477ff" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41347 } }
I20260810 18:42:22.749850 16493 raft_consensus.cc:374] T 44e2c76ae23f4724bc313aa1e51e5316 P 6a9e3573e79244e681476d93147477ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:42:22.749969 16493 raft_consensus.cc:729] T 44e2c76ae23f4724bc313aa1e51e5316 P 6a9e3573e79244e681476d93147477ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a9e3573e79244e681476d93147477ff, State: Initialized, Role: FOLLOWER
I20260810 18:42:22.750223 16493 consensus_queue.cc:260] T 44e2c76ae23f4724bc313aa1e51e5316 P 6a9e3573e79244e681476d93147477ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70f6632aa17f433ab1c0bf5502ea03a1" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44749 } } peers { permanent_uuid: "b5068707091b42a5b99875e6389e158e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40897 } } peers { permanent_uuid: "6a9e3573e79244e681476d93147477ff" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41347 } }
I20260810 18:42:22.751914 16493 ts_tablet_manager.cc:1428] T 44e2c76ae23f4724bc313aa1e51e5316 P 6a9e3573e79244e681476d93147477ff: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 18:42:22.752139 16450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:22.751584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.4.228.193:41219)
0810 18:42:22.751799 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:22.751916 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260810 18:42:22.753767 16450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:22.753278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.4.228.193:37409)
0810 18:42:22.753471 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:22.753587 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260810 18:42:22.753942 15850 heartbeater.cc:503] Master 127.4.228.253:39135 was elected leader, sending a full tablet report...
I20260810 18:42:22.830240 16491 raft_consensus.cc:482] T 44e2c76ae23f4724bc313aa1e51e5316 P b5068707091b42a5b99875e6389e158e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:42:22.830585 16491 raft_consensus.cc:504] T 44e2c76ae23f4724bc313aa1e51e5316 P b5068707091b42a5b99875e6389e158e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70f6632aa17f433ab1c0bf5502ea03a1" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44749 } } peers { permanent_uuid: "b5068707091b42a5b99875e6389e158e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40897 } } peers { permanent_uuid: "6a9e3573e79244e681476d93147477ff" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41347 } }
I20260810 18:42:22.832453 16491 leader_election.cc:288] T 44e2c76ae23f4724bc313aa1e51e5316 P b5068707091b42a5b99875e6389e158e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 70f6632aa17f433ab1c0bf5502ea03a1 (127.4.228.194:44749), 6a9e3573e79244e681476d93147477ff (127.4.228.193:41347)
I20260810 18:42:22.840389 16483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:22.833356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:58201 (local address 127.4.228.193:41347)
0810 18:42:22.833596 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:22.833604 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:22.834241 (+ 637us) server_negotiation.cc:407] Connection header received
0810 18:42:22.834331 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:22.834336 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:22.834425 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:22.834633 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:42:22.835073 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:22.836032 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:22.837943 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:22.839332 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:22.839523 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:22.839654 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:22.839940 (+ 286us) server_negotiation.cc:299] Negotiation successful
0810 18:42:22.840100 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 18:42:22.840447 16496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:22.833179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:41347 (local address 127.4.228.195:58201)
0810 18:42:22.833983 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:22.834002 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:22.834133 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:22.834606 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:22.834610 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:22.834663 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:42:22.834909 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:22.834917 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:22.836187 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:22.836194 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:22.837755 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:22.837768 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:22.839528 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:22.839535 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:22.839726 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:22.839765 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 18:42:22.839875 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 18:42:22.840012 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"mutex_wait_us":1,"thread_start_us":438,"threads_started":1}
I20260810 18:42:22.840977 16477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:22.832890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:54753 (local address 127.4.228.194:44749)
0810 18:42:22.833165 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:22.833174 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:22.833201 (+ 27us) server_negotiation.cc:407] Connection header received
0810 18:42:22.833406 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:22.833416 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:22.833529 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:22.833664 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:42:22.834310 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:22.835503 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:22.838082 (+ 2579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:22.839524 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:22.839695 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:22.839779 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:22.840540 (+ 761us) server_negotiation.cc:299] Negotiation successful
0810 18:42:22.840712 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260810 18:42:22.841454 16453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:22.832766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:44749 (local address 127.4.228.195:54753)
0810 18:42:22.833006 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:22.833025 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:22.833215 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:22.833717 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:22.833722 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:22.833756 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:42:22.834086 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:22.834101 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:22.835666 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:22.835670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:22.837847 (+ 2177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:22.837864 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:22.839818 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:22.839827 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:22.840019 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:22.840055 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 18:42:22.840159 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 18:42:22.840319 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"mutex_wait_us":54}
I20260810 18:42:22.841696 15775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e2c76ae23f4724bc313aa1e51e5316" candidate_uuid: "b5068707091b42a5b99875e6389e158e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a9e3573e79244e681476d93147477ff" is_pre_election: true
I20260810 18:42:22.842016 15775 raft_consensus.cc:2454] T 44e2c76ae23f4724bc313aa1e51e5316 P 6a9e3573e79244e681476d93147477ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5068707091b42a5b99875e6389e158e in term 0.
I20260810 18:42:22.842432 16017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e2c76ae23f4724bc313aa1e51e5316" candidate_uuid: "b5068707091b42a5b99875e6389e158e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70f6632aa17f433ab1c0bf5502ea03a1" is_pre_election: true
I20260810 18:42:22.842685 16140 leader_election.cc:302] T 44e2c76ae23f4724bc313aa1e51e5316 P b5068707091b42a5b99875e6389e158e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a9e3573e79244e681476d93147477ff, b5068707091b42a5b99875e6389e158e; no voters:
I20260810 18:42:22.842896 16017 raft_consensus.cc:2454] T 44e2c76ae23f4724bc313aa1e51e5316 P 70f6632aa17f433ab1c0bf5502ea03a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5068707091b42a5b99875e6389e158e in term 0.
I20260810 18:42:22.843271 16491 raft_consensus.cc:2781] T 44e2c76ae23f4724bc313aa1e51e5316 P b5068707091b42a5b99875e6389e158e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:42:22.843453 16491 raft_consensus.cc:482] T 44e2c76ae23f4724bc313aa1e51e5316 P b5068707091b42a5b99875e6389e158e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:42:22.843611 16491 raft_consensus.cc:3037] T 44e2c76ae23f4724bc313aa1e51e5316 P b5068707091b42a5b99875e6389e158e [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:42:22.845975 16491 raft_consensus.cc:504] T 44e2c76ae23f4724bc313aa1e51e5316 P b5068707091b42a5b99875e6389e158e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70f6632aa17f433ab1c0bf5502ea03a1" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44749 } } peers { permanent_uuid: "b5068707091b42a5b99875e6389e158e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40897 } } peers { permanent_uuid: "6a9e3573e79244e681476d93147477ff" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41347 } }
I20260810 18:42:22.846720 16491 leader_election.cc:288] T 44e2c76ae23f4724bc313aa1e51e5316 P b5068707091b42a5b99875e6389e158e [CANDIDATE]: Term 1 election: Requested vote from peers 70f6632aa17f433ab1c0bf5502ea03a1 (127.4.228.194:44749), 6a9e3573e79244e681476d93147477ff (127.4.228.193:41347)
I20260810 18:42:22.847342 15775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e2c76ae23f4724bc313aa1e51e5316" candidate_uuid: "b5068707091b42a5b99875e6389e158e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a9e3573e79244e681476d93147477ff"
I20260810 18:42:22.847339 16017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e2c76ae23f4724bc313aa1e51e5316" candidate_uuid: "b5068707091b42a5b99875e6389e158e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70f6632aa17f433ab1c0bf5502ea03a1"
I20260810 18:42:22.847563 16017 raft_consensus.cc:3037] T 44e2c76ae23f4724bc313aa1e51e5316 P 70f6632aa17f433ab1c0bf5502ea03a1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:42:22.847584 15775 raft_consensus.cc:3037] T 44e2c76ae23f4724bc313aa1e51e5316 P 6a9e3573e79244e681476d93147477ff [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:42:22.849797 15775 raft_consensus.cc:2454] T 44e2c76ae23f4724bc313aa1e51e5316 P 6a9e3573e79244e681476d93147477ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5068707091b42a5b99875e6389e158e in term 1.
I20260810 18:42:22.850325 16140 leader_election.cc:302] T 44e2c76ae23f4724bc313aa1e51e5316 P b5068707091b42a5b99875e6389e158e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a9e3573e79244e681476d93147477ff, b5068707091b42a5b99875e6389e158e; no voters:
I20260810 18:42:22.850726 16491 raft_consensus.cc:2781] T 44e2c76ae23f4724bc313aa1e51e5316 P b5068707091b42a5b99875e6389e158e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:42:22.850749 16017 raft_consensus.cc:2454] T 44e2c76ae23f4724bc313aa1e51e5316 P 70f6632aa17f433ab1c0bf5502ea03a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5068707091b42a5b99875e6389e158e in term 1.
I20260810 18:42:22.851506 16491 raft_consensus.cc:686] T 44e2c76ae23f4724bc313aa1e51e5316 P b5068707091b42a5b99875e6389e158e [term 1 LEADER]: Becoming Leader. State: Replica: b5068707091b42a5b99875e6389e158e, State: Running, Role: LEADER
I20260810 18:42:22.852291 16491 consensus_queue.cc:237] T 44e2c76ae23f4724bc313aa1e51e5316 P b5068707091b42a5b99875e6389e158e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70f6632aa17f433ab1c0bf5502ea03a1" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44749 } } peers { permanent_uuid: "b5068707091b42a5b99875e6389e158e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40897 } } peers { permanent_uuid: "6a9e3573e79244e681476d93147477ff" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41347 } }
I20260810 18:42:22.852830 16453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:22.852157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.4.228.195:44333)
0810 18:42:22.852555 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:22.852653 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243}
I20260810 18:42:22.856556 15253 catalog_manager.cc:5495] T 44e2c76ae23f4724bc313aa1e51e5316 P b5068707091b42a5b99875e6389e158e reported cstate change: term changed from 0 to 1, leader changed from <none> to b5068707091b42a5b99875e6389e158e (127.4.228.195). New cstate: current_term: 1 leader_uuid: "b5068707091b42a5b99875e6389e158e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70f6632aa17f433ab1c0bf5502ea03a1" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5068707091b42a5b99875e6389e158e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a9e3573e79244e681476d93147477ff" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41347 } health_report { overall_health: UNKNOWN } } }
I20260810 18:42:22.905550 16481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:22.899342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52796 (local address 127.4.228.195:40897)
0810 18:42:22.899552 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:22.899557 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:22.899578 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:42:22.899854 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:22.899860 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:22.899948 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:22.900115 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:42:22.900581 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:22.901407 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:22.902872 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:22.903283 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:22.903392 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:22.903851 (+ 459us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:42:22.904840 (+ 989us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:42:22.905023 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:22.905117 (+ 94us) server_negotiation.cc:299] Negotiation successful
0810 18:42:22.905214 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 18:42:22.910329 15775 raft_consensus.cc:1261] T 44e2c76ae23f4724bc313aa1e51e5316 P 6a9e3573e79244e681476d93147477ff [term 1 FOLLOWER]: Refusing update from remote peer b5068707091b42a5b99875e6389e158e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:42:22.910908 16017 raft_consensus.cc:1261] T 44e2c76ae23f4724bc313aa1e51e5316 P 70f6632aa17f433ab1c0bf5502ea03a1 [term 1 FOLLOWER]: Refusing update from remote peer b5068707091b42a5b99875e6389e158e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:42:22.911317 16497 consensus_queue.cc:1035] T 44e2c76ae23f4724bc313aa1e51e5316 P b5068707091b42a5b99875e6389e158e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a9e3573e79244e681476d93147477ff" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:42:22.911943 16491 consensus_queue.cc:1035] T 44e2c76ae23f4724bc313aa1e51e5316 P b5068707091b42a5b99875e6389e158e [LEADER]: Connected to new peer: Peer: permanent_uuid: "70f6632aa17f433ab1c0bf5502ea03a1" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:42:22.914988 16447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:22.914286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.4.228.194:45343)
0810 18:42:22.914671 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:22.914796 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230}
I20260810 18:42:22.915699 16450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:22.915156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.4.228.193:33309)
0810 18:42:22.915398 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:22.915500 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260810 18:42:22.919536 16503 mvcc.cc:205] Tried to move back new op lower bound from 7317042556552736768 to 7317042556328538112. Current Snapshot: MvccSnapshot[applied={T|T < 7317042556552736768}]
I20260810 18:42:22.920351 16447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:22.919666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.4.228.194:59181)
0810 18:42:22.919946 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:22.920076 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260810 18:42:22.935561 16450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:22.934954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.4.228.193:57473)
0810 18:42:22.935262 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:22.935362 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168}
I20260810 18:42:22.941414 16453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:22.940923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.4.228.195:50729)
0810 18:42:22.941126 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:22.941233 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260810 18:42:22.943634 16453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:22.943110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.4.228.195:53327)
0810 18:42:22.943315 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:22.943435 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260810 18:42:22.953348 15250 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38958:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:42:22.953770 15250 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38958:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:42:22.960934 15250 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 882afacf744b4b6b89526c8f0108387e: Sending DeleteTablet for 3 replicas of tablet 44e2c76ae23f4724bc313aa1e51e5316
I20260810 18:42:22.962190 15983 tablet_service.cc:1514] Processing DeleteTablet for tablet 44e2c76ae23f4724bc313aa1e51e5316 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:42:22 UTC) from {username='oryx', principal='oryx/127.4.228.253@KRBTEST.COM'} at 127.0.0.1:47140
I20260810 18:42:22.962756 16215 tablet_service.cc:1514] Processing DeleteTablet for tablet 44e2c76ae23f4724bc313aa1e51e5316 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:42:22 UTC) from {username='oryx', principal='oryx/127.4.228.253@KRBTEST.COM'} at 127.0.0.1:52790
I20260810 18:42:22.963403 15747 tablet_service.cc:1514] Processing DeleteTablet for tablet 44e2c76ae23f4724bc313aa1e51e5316 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:42:22 UTC) from {username='oryx', principal='oryx/127.4.228.253@KRBTEST.COM'} at 127.0.0.1:36890
I20260810 18:42:22.963743 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 15526
I20260810 18:42:22.966866 16521 tablet_replica.cc:330] T 44e2c76ae23f4724bc313aa1e51e5316 P 6a9e3573e79244e681476d93147477ff: stopping tablet replica
I20260810 18:42:22.967157 16521 raft_consensus.cc:2229] T 44e2c76ae23f4724bc313aa1e51e5316 P 6a9e3573e79244e681476d93147477ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:42:22.967635 16521 raft_consensus.cc:2258] T 44e2c76ae23f4724bc313aa1e51e5316 P 6a9e3573e79244e681476d93147477ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:42:22.969444 16521 ts_tablet_manager.cc:1905] T 44e2c76ae23f4724bc313aa1e51e5316 P 6a9e3573e79244e681476d93147477ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:42:22.975109 16516 tablet_replica.cc:330] T 44e2c76ae23f4724bc313aa1e51e5316 P 70f6632aa17f433ab1c0bf5502ea03a1: stopping tablet replica
I20260810 18:42:22.975170 16520 tablet_replica.cc:330] T 44e2c76ae23f4724bc313aa1e51e5316 P b5068707091b42a5b99875e6389e158e: stopping tablet replica
I20260810 18:42:22.975363 16516 raft_consensus.cc:2229] T 44e2c76ae23f4724bc313aa1e51e5316 P 70f6632aa17f433ab1c0bf5502ea03a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:42:22.975399 16520 raft_consensus.cc:2229] T 44e2c76ae23f4724bc313aa1e51e5316 P b5068707091b42a5b99875e6389e158e [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:42:22.975925 16516 raft_consensus.cc:2258] T 44e2c76ae23f4724bc313aa1e51e5316 P 70f6632aa17f433ab1c0bf5502ea03a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:42:22.976097 16520 raft_consensus.cc:2258] T 44e2c76ae23f4724bc313aa1e51e5316 P b5068707091b42a5b99875e6389e158e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:42:22.977329 16520 ts_tablet_manager.cc:1905] T 44e2c76ae23f4724bc313aa1e51e5316 P b5068707091b42a5b99875e6389e158e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:42:22.977587 16516 ts_tablet_manager.cc:1905] T 44e2c76ae23f4724bc313aa1e51e5316 P 70f6632aa17f433ab1c0bf5502ea03a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:42:22.979270 16521 ts_tablet_manager.cc:1918] T 44e2c76ae23f4724bc313aa1e51e5316 P 6a9e3573e79244e681476d93147477ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:42:22.979528 16521 log.cc:1196] T 44e2c76ae23f4724bc313aa1e51e5316 P 6a9e3573e79244e681476d93147477ff: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-0/wal/wals/44e2c76ae23f4724bc313aa1e51e5316
I20260810 18:42:22.980134 16521 ts_tablet_manager.cc:1939] T 44e2c76ae23f4724bc313aa1e51e5316 P 6a9e3573e79244e681476d93147477ff: Deleting consensus metadata
I20260810 18:42:22.981666 15235 catalog_manager.cc:4841] TS 6a9e3573e79244e681476d93147477ff (127.4.228.193:41347): tablet 44e2c76ae23f4724bc313aa1e51e5316 (table test-table [id=5adbf3346d6746fea0cb2a0d0fa78763]) successfully deleted
I20260810 18:42:22.985090 16520 ts_tablet_manager.cc:1918] T 44e2c76ae23f4724bc313aa1e51e5316 P b5068707091b42a5b99875e6389e158e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:42:22.985265 16520 log.cc:1196] T 44e2c76ae23f4724bc313aa1e51e5316 P b5068707091b42a5b99875e6389e158e: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-2/wal/wals/44e2c76ae23f4724bc313aa1e51e5316
I20260810 18:42:22.985831 16520 ts_tablet_manager.cc:1939] T 44e2c76ae23f4724bc313aa1e51e5316 P b5068707091b42a5b99875e6389e158e: Deleting consensus metadata
I20260810 18:42:22.987183 15235 catalog_manager.cc:4841] TS b5068707091b42a5b99875e6389e158e (127.4.228.195:40897): tablet 44e2c76ae23f4724bc313aa1e51e5316 (table test-table [id=5adbf3346d6746fea0cb2a0d0fa78763]) successfully deleted
I20260810 18:42:22.992760 16516 ts_tablet_manager.cc:1918] T 44e2c76ae23f4724bc313aa1e51e5316 P 70f6632aa17f433ab1c0bf5502ea03a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:42:22.993028 16516 log.cc:1196] T 44e2c76ae23f4724bc313aa1e51e5316 P 70f6632aa17f433ab1c0bf5502ea03a1: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387267874959-5011-0/minicluster-data/ts-1/wal/wals/44e2c76ae23f4724bc313aa1e51e5316
I20260810 18:42:22.993571 16516 ts_tablet_manager.cc:1939] T 44e2c76ae23f4724bc313aa1e51e5316 P 70f6632aa17f433ab1c0bf5502ea03a1: Deleting consensus metadata
I20260810 18:42:22.995398 15235 catalog_manager.cc:4841] TS 70f6632aa17f433ab1c0bf5502ea03a1 (127.4.228.194:44749): tablet 44e2c76ae23f4724bc313aa1e51e5316 (table test-table [id=5adbf3346d6746fea0cb2a0d0fa78763]) successfully deleted
I20260810 18:42:23.001667 16483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:22.965238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36892 (local address 127.4.228.193:41347)
0810 18:42:22.965457 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:22.965465 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:22.975103 (+ 9638us) server_negotiation.cc:407] Connection header received
0810 18:42:22.975359 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:22.975365 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:22.975459 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:22.975599 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:22.978066 (+ 2467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:22.979279 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:22.981373 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:22.982019 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:22.982167 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:22.994152 (+ 11985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:22.994182 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:22.994208 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:22.994265 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:22.997771 (+ 3506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:22.999320 (+ 1549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:22.999330 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:22.999332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:22.999494 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:22.999919 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:22.999930 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:22.999932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:23.000335 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:23.000680 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:23.000993 (+ 313us) server_negotiation.cc:299] Negotiation successful
0810 18:42:23.001233 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 18:42:23.002403 15839 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:42:23.098254 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 15526
I20260810 18:42:23.124245 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 15876
I20260810 18:42:23.138805 16477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:23.125108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47144 (local address 127.4.228.194:44749)
0810 18:42:23.125324 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:23.125331 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:23.125438 (+ 107us) server_negotiation.cc:407] Connection header received
0810 18:42:23.125656 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:23.125662 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:23.125747 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:23.125872 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:23.127331 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:23.128197 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:23.129454 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:23.129863 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:23.130000 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:23.132874 (+ 2874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:23.132890 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:23.132909 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:23.132942 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:23.135910 (+ 2968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:23.136622 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:23.136631 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:23.136635 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:23.136781 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:23.137171 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:23.137176 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:23.137180 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:23.137555 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:23.137870 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:23.138183 (+ 313us) server_negotiation.cc:299] Negotiation successful
0810 18:42:23.138419 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260810 18:42:23.139657 16090 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:42:23.165974 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:23.165396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.0.0.1:39640)
0810 18:42:23.165629 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:23.165753 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260810 18:42:23.167330 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:23.166871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45361 (local address 127.0.0.1:39652)
0810 18:42:23.167065 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:23.167158 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260810 18:42:23.265889 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 15876
I20260810 18:42:23.317028 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 16117
I20260810 18:42:23.335810 16481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:23.317995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39944 (local address 127.4.228.195:40897)
0810 18:42:23.318254 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:23.318264 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:23.318335 (+ 71us) server_negotiation.cc:407] Connection header received
0810 18:42:23.318647 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:23.318654 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:23.318761 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:23.318892 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:23.320390 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:23.321302 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:23.323367 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:23.323840 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:23.323955 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:23.328155 (+ 4200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:23.328179 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:23.328200 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:23.328254 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:23.332052 (+ 3798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:23.333149 (+ 1097us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:23.333190 (+ 41us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:23.333194 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:23.333396 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:23.333910 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:23.333914 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:23.333916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:23.334344 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:23.334711 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:23.335246 (+ 535us) server_negotiation.cc:299] Negotiation successful
0810 18:42:23.335436 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260810 18:42:23.336781 16310 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:42:23.474789 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 16117
I20260810 18:42:23.507231 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 15034
I20260810 18:42:23.508656 15306 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:42:23.636533 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 15034
W20260810 18:42:23.669461 15541 connection.cc:535] server connection from 127.0.0.1:34682 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 18:42:23.670564 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 15202
I20260810 18:42:23.672168 15617 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:42:23.801852 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 15202
2026-08-10T18:42:23Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (13042 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:42:23 dist-test-slave-7kzw krb5kdc[16534](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:42:23 dist-test-slave-7kzw krb5kdc[16534](info): set up 2 sockets
Aug 10 18:42:23 dist-test-slave-7kzw krb5kdc[16534](info): commencing operation
krb5kdc: starting...
W20260810 18:42:25.235206 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.373s user 0.002s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:42:25 dist-test-slave-7kzw krb5kdc[16534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:42:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:42:25Z Disabled control of system clock
WARNING: no policy specified for oryx/127.4.228.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.228.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.228.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.254.keytab.
Entry for principal oryx/127.4.228.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.254.keytab.
I20260810 18:42:25.543574 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:33641
--webserver_interface=127.4.228.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.254.keytab
--principal=oryx/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:40997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:33641
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:42:25.920580 16552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:42:25.920934 16552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:42:25.921082 16552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:42:25.928745 16552 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:42:25.928869 16552 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 18:42:25.928951 16552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:42:25.929049 16552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:42:25.929143 16552 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:42:25.929244 16552 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:42:25.940284 16552 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:40997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:33641
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.254.keytab
--principal=oryx/127.4.228.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:33641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:42:25.941861 16552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:42:25.943399 16552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:42:25.956501 16559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:25.956471 16562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:25.956547 16558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:42:25.957435 16552 server_base.cc:1034] running on GCE node
I20260810 18:42:25.958449 16552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:42:25.960358 16552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:42:25.961737 16552 hybrid_clock.cc:648] HybridClock initialized: now 1786387345961614 us; error 89 us; skew 500 ppm
I20260810 18:42:25.962219 16552 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:42:25 dist-test-slave-7kzw krb5kdc[16534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387345, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:42:25.966306 16552 init.cc:377] Logged in from keytab as oryx/127.4.228.254@KRBTEST.COM (short username oryx)
I20260810 18:42:25.968256 16552 webserver.cc:460] Webserver started at http://127.4.228.254:41185/ using document root <none> and password file <none>
I20260810 18:42:25.969162 16552 fs_manager.cc:359] Metadata directory not provided
I20260810 18:42:25.969324 16552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:42:25.969702 16552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:42:25.973042 16552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "8cfbe048354045c5b31fc749e0cfe06e"
format_stamp: "Formatted at 2026-08-10 18:42:25 on dist-test-slave-7kzw"
server_key: "374075218dbf613c6fc3ce54c9176460"
server_key_iv: "f9b6841065873c7153bba4bc7b782301"
server_key_version: "encryptionkey@0"
I20260810 18:42:25.973915 16552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "8cfbe048354045c5b31fc749e0cfe06e"
format_stamp: "Formatted at 2026-08-10 18:42:25 on dist-test-slave-7kzw"
server_key: "374075218dbf613c6fc3ce54c9176460"
server_key_iv: "f9b6841065873c7153bba4bc7b782301"
server_key_version: "encryptionkey@0"
I20260810 18:42:25.980072 16552 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260810 18:42:25.984491 16573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:25.986209 16552 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:42:25.986562 16552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "8cfbe048354045c5b31fc749e0cfe06e"
format_stamp: "Formatted at 2026-08-10 18:42:25 on dist-test-slave-7kzw"
server_key: "374075218dbf613c6fc3ce54c9176460"
server_key_iv: "f9b6841065873c7153bba4bc7b782301"
server_key_version: "encryptionkey@0"
I20260810 18:42:25.986855 16552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:42:26.009310 16552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:42:26.015420 16552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:42:26.015756 16552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:42:26.043349 16552 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:33641
I20260810 18:42:26.043370 16664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:33641 every 8 connection(s)
I20260810 18:42:26.044983 16552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:42:26.046340 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 16552
I20260810 18:42:26.046682 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:42:26.047775 5011 external_mini_cluster.cc:1428] Setting key 1d6a5f0ba7954b1645e9e47ee33d4e4a
I20260810 18:42:26.051083 16665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 18:42:26 dist-test-slave-7kzw krb5kdc[16534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM
I20260810 18:42:26.067888 16665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cfbe048354045c5b31fc749e0cfe06e: Bootstrap starting.
I20260810 18:42:26.072348 16665 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8cfbe048354045c5b31fc749e0cfe06e: Neither blocks nor log segments found. Creating new log.
I20260810 18:42:26.073698 16665 log.cc:824] T 00000000000000000000000000000000 P 8cfbe048354045c5b31fc749e0cfe06e: Log is configured to *not* fsync() on all Append() calls
I20260810 18:42:26.076682 16669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:26.050355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60456 (local address 127.4.228.254:33641)
0810 18:42:26.051397 (+ 1042us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:26.051410 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:26.051456 (+ 46us) server_negotiation.cc:407] Connection header received
0810 18:42:26.053281 (+ 1825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:26.053305 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:26.053732 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:26.054112 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:26.056487 (+ 2375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:26.057939 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:26.060188 (+ 2249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:26.060929 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:26.061167 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:26.067408 (+ 6241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:26.067419 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:26.067439 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:26.067491 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:26.071479 (+ 3988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:26.072396 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:26.072405 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:26.072412 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:26.072695 (+ 283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:26.073165 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:26.073171 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:26.073175 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:26.073811 (+ 636us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:26.074184 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:26.074853 (+ 669us) server_negotiation.cc:299] Negotiation successful
0810 18:42:26.075418 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":802,"thread_start_us":458,"threads_started":1}
I20260810 18:42:26.077376 16665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cfbe048354045c5b31fc749e0cfe06e: No bootstrap required, opened a new log
I20260810 18:42:26.086113 16665 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8cfbe048354045c5b31fc749e0cfe06e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cfbe048354045c5b31fc749e0cfe06e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 33641 } }
I20260810 18:42:26.086570 16665 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8cfbe048354045c5b31fc749e0cfe06e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:42:26.086699 16665 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8cfbe048354045c5b31fc749e0cfe06e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cfbe048354045c5b31fc749e0cfe06e, State: Initialized, Role: FOLLOWER
I20260810 18:42:26.087323 16665 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cfbe048354045c5b31fc749e0cfe06e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cfbe048354045c5b31fc749e0cfe06e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 33641 } }
I20260810 18:42:26.087592 16665 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8cfbe048354045c5b31fc749e0cfe06e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:42:26.087750 16665 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cfbe048354045c5b31fc749e0cfe06e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:42:26.087961 16665 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8cfbe048354045c5b31fc749e0cfe06e [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:42:26.090189 16665 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cfbe048354045c5b31fc749e0cfe06e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cfbe048354045c5b31fc749e0cfe06e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 33641 } }
I20260810 18:42:26.090813 16665 leader_election.cc:302] T 00000000000000000000000000000000 P 8cfbe048354045c5b31fc749e0cfe06e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8cfbe048354045c5b31fc749e0cfe06e; no voters:
I20260810 18:42:26.091480 16665 leader_election.cc:288] T 00000000000000000000000000000000 P 8cfbe048354045c5b31fc749e0cfe06e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:42:26.091682 16671 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cfbe048354045c5b31fc749e0cfe06e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:42:26.093528 16665 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8cfbe048354045c5b31fc749e0cfe06e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:42:26.095082 16671 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8cfbe048354045c5b31fc749e0cfe06e [term 1 LEADER]: Becoming Leader. State: Replica: 8cfbe048354045c5b31fc749e0cfe06e, State: Running, Role: LEADER
I20260810 18:42:26.095983 16671 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cfbe048354045c5b31fc749e0cfe06e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cfbe048354045c5b31fc749e0cfe06e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 33641 } }
I20260810 18:42:26.099834 16673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cfbe048354045c5b31fc749e0cfe06e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cfbe048354045c5b31fc749e0cfe06e. Latest consensus state: current_term: 1 leader_uuid: "8cfbe048354045c5b31fc749e0cfe06e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cfbe048354045c5b31fc749e0cfe06e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 33641 } } }
I20260810 18:42:26.100898 16673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cfbe048354045c5b31fc749e0cfe06e [sys.catalog]: This master's current role is: LEADER
I20260810 18:42:26.100406 16672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cfbe048354045c5b31fc749e0cfe06e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8cfbe048354045c5b31fc749e0cfe06e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cfbe048354045c5b31fc749e0cfe06e" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 33641 } } }
I20260810 18:42:26.101455 16672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cfbe048354045c5b31fc749e0cfe06e [sys.catalog]: This master's current role is: LEADER
W20260810 18:42:26.105602 16691 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8cfbe048354045c5b31fc749e0cfe06e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:42:26.105826 16691 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:42:26.106031 16694 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:42:26.107968 16694 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:42:26.118111 16694 catalog_manager.cc:1347] Generated new cluster ID: c33ff8ab429545b49fed427cd81afe2c
I20260810 18:42:26.118269 16694 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:42:26.137617 16694 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:42:26.139241 16694 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:42:26.150807 16694 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8cfbe048354045c5b31fc749e0cfe06e: Generated new TSK 0
I20260810 18:42:26.151734 16694 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.4.228.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.228.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.228.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.193.keytab.
Entry for principal oryx/127.4.228.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.193.keytab.
I20260810 18:42:26.237087 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.193:0
--local_ip_for_outbound_sockets=127.4.228.193
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:33641
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.193.keytab
--principal=oryx/127.4.228.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:40997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:42:26.626716 16703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:42:26.627056 16703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:42:26.627221 16703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:42:26.635087 16703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:42:26.635231 16703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:42:26.635380 16703 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 18:42:26.635490 16703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.193
I20260810 18:42:26.645756 16703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:40997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.193.keytab
--principal=oryx/127.4.228.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.228.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.228.254:33641
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.193
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:42:26.647243 16703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:42:26.648727 16703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:42:26.664073 16711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:26.664340 16712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:26.664683 16716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:42:26.665290 16703 server_base.cc:1034] running on GCE node
I20260810 18:42:26.666009 16703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:42:26.667116 16703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:42:26.668393 16703 hybrid_clock.cc:648] HybridClock initialized: now 1786387346668324 us; error 38 us; skew 500 ppm
I20260810 18:42:26.668746 16703 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:42:26 dist-test-slave-7kzw krb5kdc[16534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387346, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:42:26.672281 16703 init.cc:377] Logged in from keytab as oryx/127.4.228.193@KRBTEST.COM (short username oryx)
I20260810 18:42:26.673964 16703 webserver.cc:460] Webserver started at http://127.4.228.193:44683/ using document root <none> and password file <none>
I20260810 18:42:26.674832 16703 fs_manager.cc:359] Metadata directory not provided
I20260810 18:42:26.674953 16703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:42:26.675321 16703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:42:26.678542 16703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-0/data/instance:
uuid: "baa8dad45c5f493d92d140dea29ec5f9"
format_stamp: "Formatted at 2026-08-10 18:42:26 on dist-test-slave-7kzw"
server_key: "cc48ef95c4472f9ca2aa6c399288b6e7"
server_key_iv: "d888fe9d5c9168e1d9552b76849ac9b7"
server_key_version: "encryptionkey@0"
I20260810 18:42:26.679416 16703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance:
uuid: "baa8dad45c5f493d92d140dea29ec5f9"
format_stamp: "Formatted at 2026-08-10 18:42:26 on dist-test-slave-7kzw"
server_key: "cc48ef95c4472f9ca2aa6c399288b6e7"
server_key_iv: "d888fe9d5c9168e1d9552b76849ac9b7"
server_key_version: "encryptionkey@0"
I20260810 18:42:26.685196 16703 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 18:42:26.689373 16727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:26.691190 16703 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260810 18:42:26.691394 16703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-0/wal
uuid: "baa8dad45c5f493d92d140dea29ec5f9"
format_stamp: "Formatted at 2026-08-10 18:42:26 on dist-test-slave-7kzw"
server_key: "cc48ef95c4472f9ca2aa6c399288b6e7"
server_key_iv: "d888fe9d5c9168e1d9552b76849ac9b7"
server_key_version: "encryptionkey@0"
I20260810 18:42:26.691663 16703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:42:26.721995 16703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:42:26.727304 16703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:42:26.727663 16703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:42:26.730729 16703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:42:26.730851 16703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:26.731022 16703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:42:26.731125 16703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 18:42:26 dist-test-slave-7kzw krb5kdc[16534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387346, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM
I20260810 18:42:26.762322 16739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:26.732341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:33641 (local address 127.4.228.193:34171)
0810 18:42:26.733326 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:26.733396 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:26.735665 (+ 2269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:26.736913 (+ 1248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:26.736925 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:26.737887 (+ 962us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:26.738883 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:26.738935 (+ 52us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:26.740803 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:26.740812 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:26.743028 (+ 2216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:26.743053 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:26.744386 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:26.744400 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:26.744667 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:26.745544 (+ 877us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:26.745600 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:26.749659 (+ 4059us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:26.754886 (+ 5227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:26.754907 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:26.754941 (+ 34us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:26.755934 (+ 993us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:26.756583 (+ 649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:26.756587 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:26.756590 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:26.756971 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:26.757858 (+ 887us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:26.757875 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:26.758630 (+ 755us) client_negotiation.cc:769] Sending connection context
0810 18:42:26.759068 (+ 438us) client_negotiation.cc:240] Negotiation successful
0810 18:42:26.759971 (+ 903us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"thread_start_us":420,"threads_started":1}
I20260810 18:42:26.763242 16738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:26.731683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:34171 (local address 127.4.228.254:33641)
0810 18:42:26.732293 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:26.732300 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:26.733557 (+ 1257us) server_negotiation.cc:407] Connection header received
0810 18:42:26.736085 (+ 2528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:26.736091 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:26.736187 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:26.736308 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:26.739177 (+ 2869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:26.740552 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:26.743281 (+ 2729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:26.744071 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:26.744250 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:26.750073 (+ 5823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:26.750084 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:26.750091 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:26.750146 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:26.754660 (+ 4514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:26.756137 (+ 1477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:26.756147 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:26.756152 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:26.756380 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:26.757172 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:26.757182 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:26.757187 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:26.757627 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:26.757847 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:26.762382 (+ 4535us) server_negotiation.cc:299] Negotiation successful
0810 18:42:26.762747 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":335,"threads_started":1}
I20260810 18:42:26.782303 16703 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.193:35775
I20260810 18:42:26.782374 16905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.193:35775 every 8 connection(s)
I20260810 18:42:26.784078 16703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
I20260810 18:42:26.787899 16906 heartbeater.cc:347] Connected to a master server at 127.4.228.254:33641
I20260810 18:42:26.788244 16906 heartbeater.cc:464] Registering TS with master...
I20260810 18:42:26.789268 16906 heartbeater.cc:511] Master 127.4.228.254:33641 requested a full tablet report, sending...
I20260810 18:42:26.791105 16738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:26.772794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:58713 (local address 127.4.228.254:33641)
0810 18:42:26.773067 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:26.773077 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:26.773919 (+ 842us) server_negotiation.cc:407] Connection header received
0810 18:42:26.774132 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:26.774140 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:26.774274 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:26.774411 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:26.776242 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:26.777653 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:26.780590 (+ 2937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:26.781269 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:26.782631 (+ 1362us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:26.783881 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:26.783891 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:26.783899 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:26.783953 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:26.787650 (+ 3697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:26.788649 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:26.788653 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:26.788656 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:26.788834 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:26.789373 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:26.789381 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:26.789384 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:26.789720 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:26.789841 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:26.790387 (+ 546us) server_negotiation.cc:299] Negotiation successful
0810 18:42:26.790671 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 18:42:26.791340 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 16703
I20260810 18:42:26.791571 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance
I20260810 18:42:26.792091 16599 ts_manager.cc:194] Registered new tserver with Master: baa8dad45c5f493d92d140dea29ec5f9 (127.4.228.193:35775)
I20260810 18:42:26.792330 5011 external_mini_cluster.cc:1428] Setting key e662c5bfee6d05b688804613b8a29ccd
I20260810 18:42:26.794100 16599 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.228.193@KRBTEST.COM'} at 127.4.228.193:34171
I20260810 18:42:26.794238 16880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:26.772962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:33641 (local address 127.4.228.193:58713)
0810 18:42:26.773691 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:26.773719 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:26.773920 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:26.774504 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:26.774511 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:26.775373 (+ 862us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:26.776001 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:26.776023 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:26.777886 (+ 1863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:26.777895 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:26.780291 (+ 2396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:26.780313 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:26.781537 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:26.781543 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:26.781800 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:26.782820 (+ 1020us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:26.782839 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:26.783633 (+ 794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:26.787837 (+ 4204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:26.787845 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:26.787852 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:26.788512 (+ 660us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:26.788958 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:26.788963 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:26.788967 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:26.789187 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:26.789850 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:26.789855 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:26.790082 (+ 227us) client_negotiation.cc:769] Sending connection context
0810 18:42:26.790238 (+ 156us) client_negotiation.cc:240] Negotiation successful
0810 18:42:26.790630 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":391,"threads_started":1}
WARNING: no policy specified for oryx/127.4.228.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.228.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.228.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.194.keytab.
Entry for principal oryx/127.4.228.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.194.keytab.
I20260810 18:42:26.865581 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.194:0
--local_ip_for_outbound_sockets=127.4.228.194
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:33641
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.194.keytab
--principal=oryx/127.4.228.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:40997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:42:27.255398 16911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:42:27.255740 16911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:42:27.255923 16911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:42:27.263716 16911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:42:27.263849 16911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:42:27.263985 16911 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 18:42:27.264122 16911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.194
I20260810 18:42:27.274772 16911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:40997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.194.keytab
--principal=oryx/127.4.228.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.228.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.228.254:33641
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.194
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:42:27.276242 16911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:42:27.277714 16911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:42:27.295501 16919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:27.295675 16921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:27.295859 16925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:42:27.296768 16911 server_base.cc:1034] running on GCE node
I20260810 18:42:27.297533 16911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:42:27.299080 16911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:42:27.300383 16911 hybrid_clock.cc:648] HybridClock initialized: now 1786387347300314 us; error 46 us; skew 500 ppm
I20260810 18:42:27.300693 16911 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:42:27 dist-test-slave-7kzw krb5kdc[16534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387347, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:42:27.304593 16911 init.cc:377] Logged in from keytab as oryx/127.4.228.194@KRBTEST.COM (short username oryx)
I20260810 18:42:27.306831 16911 webserver.cc:460] Webserver started at http://127.4.228.194:41915/ using document root <none> and password file <none>
I20260810 18:42:27.307698 16911 fs_manager.cc:359] Metadata directory not provided
I20260810 18:42:27.307859 16911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:42:27.308243 16911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:42:27.311604 16911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-1/data/instance:
uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc"
format_stamp: "Formatted at 2026-08-10 18:42:27 on dist-test-slave-7kzw"
server_key: "81e652c632ae10b4b8ce51aacb24a740"
server_key_iv: "259a4ad41e89edb0c0231366c408059d"
server_key_version: "encryptionkey@0"
I20260810 18:42:27.312523 16911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance:
uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc"
format_stamp: "Formatted at 2026-08-10 18:42:27 on dist-test-slave-7kzw"
server_key: "81e652c632ae10b4b8ce51aacb24a740"
server_key_iv: "259a4ad41e89edb0c0231366c408059d"
server_key_version: "encryptionkey@0"
I20260810 18:42:27.319079 16911 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260810 18:42:27.323154 16936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:27.324774 16911 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:42:27.324990 16911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-1/wal
uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc"
format_stamp: "Formatted at 2026-08-10 18:42:27 on dist-test-slave-7kzw"
server_key: "81e652c632ae10b4b8ce51aacb24a740"
server_key_iv: "259a4ad41e89edb0c0231366c408059d"
server_key_version: "encryptionkey@0"
I20260810 18:42:27.325253 16911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:42:27.342409 16911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:42:27.347368 16911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:42:27.347692 16911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:42:27.350873 16911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:42:27.351003 16911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:27.351186 16911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:42:27.351274 16911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 18:42:27 dist-test-slave-7kzw krb5kdc[16534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387347, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM
I20260810 18:42:27.377040 16949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:27.351649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:42541 (local address 127.4.228.254:33641)
0810 18:42:27.352406 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:27.352413 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:27.353716 (+ 1303us) server_negotiation.cc:407] Connection header received
0810 18:42:27.356065 (+ 2349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:27.356071 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:27.356163 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:27.356327 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:27.358909 (+ 2582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:27.359758 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:27.361577 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:27.362024 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:27.362219 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:27.367960 (+ 5741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:27.367971 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:27.367981 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:27.368045 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:27.372293 (+ 4248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:27.373400 (+ 1107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:27.373409 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:27.373415 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:27.373612 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:27.374345 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:27.374354 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:27.374360 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:27.374854 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:27.375061 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:27.376024 (+ 963us) server_negotiation.cc:299] Negotiation successful
0810 18:42:27.376391 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":464,"threads_started":1}
I20260810 18:42:27.377605 16952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:27.352442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:33641 (local address 127.4.228.194:42541)
0810 18:42:27.353474 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:27.353531 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:27.355675 (+ 2144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:27.356629 (+ 954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:27.356644 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:27.357590 (+ 946us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:27.358683 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:27.358710 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:27.359988 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:27.359992 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:27.361409 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:27.361424 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:27.362297 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:27.362306 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:27.362649 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:27.363627 (+ 978us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:27.363654 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:27.367605 (+ 3951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:27.372542 (+ 4937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:27.372551 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:27.372571 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:27.373217 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:27.373779 (+ 562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:27.373784 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:27.373788 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:27.374145 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:27.375037 (+ 892us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:27.375049 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:27.375483 (+ 434us) client_negotiation.cc:769] Sending connection context
0810 18:42:27.375773 (+ 290us) client_negotiation.cc:240] Negotiation successful
0810 18:42:27.376243 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"thread_start_us":419,"threads_started":1}
I20260810 18:42:27.404927 16949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:27.384871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:50965 (local address 127.4.228.254:33641)
0810 18:42:27.385309 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:27.385318 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:27.385885 (+ 567us) server_negotiation.cc:407] Connection header received
0810 18:42:27.386208 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:27.386213 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:27.386294 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:27.386415 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:27.389247 (+ 2832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:27.390176 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:27.392933 (+ 2757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:27.393662 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:27.393848 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:27.396960 (+ 3112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:27.396966 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:27.396971 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:27.397013 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:27.401183 (+ 4170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:27.402038 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:27.402053 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:27.402057 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:27.402214 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:27.402803 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:27.402810 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:27.402815 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:27.403404 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:27.403603 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:27.404144 (+ 541us) server_negotiation.cc:299] Negotiation successful
0810 18:42:27.404405 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249}
I20260810 18:42:27.405042 17040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:27.384910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:33641 (local address 127.4.228.194:50965)
0810 18:42:27.385726 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:27.385755 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:27.385971 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:27.386616 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:27.386628 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:27.388020 (+ 1392us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:27.388900 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:27.388928 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:27.390465 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:27.390471 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:27.392652 (+ 2181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:27.392674 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:27.393921 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:27.393929 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:27.394143 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:27.395049 (+ 906us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:27.395113 (+ 64us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:27.396640 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:27.401359 (+ 4719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:27.401366 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:27.401371 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:27.401882 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:27.402344 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:27.402349 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:27.402353 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:27.402628 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:27.403584 (+ 956us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:27.403595 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:27.403842 (+ 247us) client_negotiation.cc:769] Sending connection context
0810 18:42:27.404015 (+ 173us) client_negotiation.cc:240] Negotiation successful
0810 18:42:27.404405 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":427,"threads_started":1}
I20260810 18:42:27.414839 16911 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.194:43603
I20260810 18:42:27.414867 17126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.194:43603 every 8 connection(s)
I20260810 18:42:27.416211 16911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
I20260810 18:42:27.418787 17127 heartbeater.cc:347] Connected to a master server at 127.4.228.254:33641
I20260810 18:42:27.419057 17127 heartbeater.cc:464] Registering TS with master...
I20260810 18:42:27.419763 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 16911
I20260810 18:42:27.419816 17127 heartbeater.cc:511] Master 127.4.228.254:33641 requested a full tablet report, sending...
I20260810 18:42:27.419983 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance
I20260810 18:42:27.420485 5011 external_mini_cluster.cc:1428] Setting key abcc78ec18843a9e92e47b80e10e8d6a
I20260810 18:42:27.421494 16596 ts_manager.cc:194] Registered new tserver with Master: 6e5373d9feb843cf8ef92ae3e7dbb5bc (127.4.228.194:43603)
I20260810 18:42:27.422766 16596 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.228.194@KRBTEST.COM'} at 127.4.228.194:42541
WARNING: no policy specified for oryx/127.4.228.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.228.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.228.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.195.keytab.
Entry for principal oryx/127.4.228.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.195.keytab.
I20260810 18:42:27.492790 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.195:0
--local_ip_for_outbound_sockets=127.4.228.195
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:33641
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.195.keytab
--principal=oryx/127.4.228.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:40997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:42:27.798931 16906 heartbeater.cc:503] Master 127.4.228.254:33641 was elected leader, sending a full tablet report...
W20260810 18:42:27.868073 17131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:42:27.868464 17131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:42:27.868649 17131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:42:27.876483 17131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:42:27.876706 17131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:42:27.876886 17131 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 18:42:27.877044 17131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.195
I20260810 18:42:27.888866 17131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:40997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.195.keytab
--principal=oryx/127.4.228.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.228.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.228.254:33641
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.195
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:42:27.890949 17131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:42:27.892642 17131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:42:27.907725 17137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:27.907862 17138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:27.907887 17141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:42:27.909065 17131 server_base.cc:1034] running on GCE node
I20260810 18:42:27.909910 17131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:42:27.911087 17131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:42:27.912395 17131 hybrid_clock.cc:648] HybridClock initialized: now 1786387347912331 us; error 33 us; skew 500 ppm
I20260810 18:42:27.912789 17131 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:42:27 dist-test-slave-7kzw krb5kdc[16534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387347, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:42:27.916400 17131 init.cc:377] Logged in from keytab as oryx/127.4.228.195@KRBTEST.COM (short username oryx)
I20260810 18:42:27.918233 17131 webserver.cc:460] Webserver started at http://127.4.228.195:45831/ using document root <none> and password file <none>
I20260810 18:42:27.919297 17131 fs_manager.cc:359] Metadata directory not provided
I20260810 18:42:27.919572 17131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:42:27.920276 17131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:42:27.923659 17131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-2/data/instance:
uuid: "0d3eb3675e6543c98972fe7cf6078944"
format_stamp: "Formatted at 2026-08-10 18:42:27 on dist-test-slave-7kzw"
server_key: "8cd0d82bc0fd97e5da0e6aa8ae3b989e"
server_key_iv: "d48bc0c773959fca0c39a20e6f7c68e7"
server_key_version: "encryptionkey@0"
I20260810 18:42:27.924533 17131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance:
uuid: "0d3eb3675e6543c98972fe7cf6078944"
format_stamp: "Formatted at 2026-08-10 18:42:27 on dist-test-slave-7kzw"
server_key: "8cd0d82bc0fd97e5da0e6aa8ae3b989e"
server_key_iv: "d48bc0c773959fca0c39a20e6f7c68e7"
server_key_version: "encryptionkey@0"
I20260810 18:42:27.930563 17131 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260810 18:42:27.934422 17151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:27.936035 17131 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260810 18:42:27.936270 17131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-2/wal
uuid: "0d3eb3675e6543c98972fe7cf6078944"
format_stamp: "Formatted at 2026-08-10 18:42:27 on dist-test-slave-7kzw"
server_key: "8cd0d82bc0fd97e5da0e6aa8ae3b989e"
server_key_iv: "d48bc0c773959fca0c39a20e6f7c68e7"
server_key_version: "encryptionkey@0"
I20260810 18:42:27.936525 17131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:42:27.957090 17131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:42:27.962255 17131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:42:27.962648 17131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:42:27.965809 17131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:42:27.965927 17131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:27.966140 17131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:42:27.966321 17131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 18:42:27 dist-test-slave-7kzw krb5kdc[16534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387347, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.195@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM
I20260810 18:42:28.004307 17163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:27.966891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:53663 (local address 127.4.228.254:33641)
0810 18:42:27.967450 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:27.967461 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:27.968853 (+ 1392us) server_negotiation.cc:407] Connection header received
0810 18:42:27.972582 (+ 3729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:27.972588 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:27.972701 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:27.972839 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:27.975716 (+ 2877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:27.976839 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:27.980901 (+ 4062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:27.981566 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:27.981740 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:27.988810 (+ 7070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:27.988819 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:27.988825 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:27.988882 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:27.993807 (+ 4925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:28.000198 (+ 6391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:28.000206 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:28.000212 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:28.000414 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:28.001248 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:28.001257 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:28.001264 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:28.001872 (+ 608us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:28.002095 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:28.003257 (+ 1162us) server_negotiation.cc:299] Negotiation successful
0810 18:42:28.003671 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":238,"threads_started":1}
I20260810 18:42:28.005172 17164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:27.967608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:33641 (local address 127.4.228.195:53663)
0810 18:42:27.968598 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:27.968680 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:27.972082 (+ 3402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:27.973391 (+ 1309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:27.973410 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:27.974353 (+ 943us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:27.975478 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:27.975502 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:27.977065 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:27.977071 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:27.980582 (+ 3511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:27.980609 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:27.981927 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:27.981937 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:27.982345 (+ 408us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:27.983486 (+ 1141us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:27.983536 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:27.988278 (+ 4742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:27.994212 (+ 5934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:27.994234 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:27.994265 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:27.999926 (+ 5661us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:28.000581 (+ 655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:28.000592 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:28.000597 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:28.001015 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:28.002081 (+ 1066us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:28.002097 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:28.002696 (+ 599us) client_negotiation.cc:769] Sending connection context
0810 18:42:28.002996 (+ 300us) client_negotiation.cc:240] Negotiation successful
0810 18:42:28.003749 (+ 753us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":355,"threads_started":1}
I20260810 18:42:28.024749 17131 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.195:36567
I20260810 18:42:28.024788 17329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.195:36567 every 8 connection(s)
I20260810 18:42:28.027124 17131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
I20260810 18:42:28.029995 17333 heartbeater.cc:347] Connected to a master server at 127.4.228.254:33641
I20260810 18:42:28.030288 17333 heartbeater.cc:464] Registering TS with master...
I20260810 18:42:28.031266 17333 heartbeater.cc:511] Master 127.4.228.254:33641 requested a full tablet report, sending...
I20260810 18:42:28.033090 16596 ts_manager.cc:194] Registered new tserver with Master: 0d3eb3675e6543c98972fe7cf6078944 (127.4.228.195:36567)
I20260810 18:42:28.034037 16596 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.228.195@KRBTEST.COM'} at 127.4.228.195:53663
I20260810 18:42:28.034112 17163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:28.014677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:58659 (local address 127.4.228.254:33641)
0810 18:42:28.014926 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:28.014935 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:28.016191 (+ 1256us) server_negotiation.cc:407] Connection header received
0810 18:42:28.016588 (+ 397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:28.016595 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:28.016919 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:28.017231 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:28.019632 (+ 2401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:28.020910 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:28.023497 (+ 2587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:28.024232 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:28.024385 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:28.026988 (+ 2603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:28.026994 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:28.026998 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:28.027036 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:28.030773 (+ 3737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:28.031743 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:28.031748 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:28.031751 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:28.031891 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:28.032347 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:28.032352 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:28.032355 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:28.032795 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:28.032961 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:28.033453 (+ 492us) server_negotiation.cc:299] Negotiation successful
0810 18:42:28.033756 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 18:42:28.034162 17298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:28.014521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:33641 (local address 127.4.228.195:58659)
0810 18:42:28.016039 (+ 1518us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:28.016065 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:28.016363 (+ 298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:28.017555 (+ 1192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:28.017563 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:28.018562 (+ 999us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:28.019372 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:28.019397 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:28.021153 (+ 1756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:28.021162 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:28.023288 (+ 2126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:28.023306 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:28.024492 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:28.024504 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:28.024716 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:28.025512 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:28.025538 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:28.026703 (+ 1165us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:28.030922 (+ 4219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:28.030927 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:28.030931 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:28.031589 (+ 658us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:28.032022 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:28.032026 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:28.032030 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:28.032215 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:28.032932 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:28.032936 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:28.033098 (+ 162us) client_negotiation.cc:769] Sending connection context
0810 18:42:28.033331 (+ 233us) client_negotiation.cc:240] Negotiation successful
0810 18:42:28.033689 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1282,"thread_start_us":1056,"threads_started":1}
I20260810 18:42:28.034832 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 17131
I20260810 18:42:28.035044 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance
I20260810 18:42:28.035602 5011 external_mini_cluster.cc:1428] Setting key a6faf201ead7bdcff02440828411b2b4
I20260810 18:42:28.039642 5011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 18:42:28 dist-test-slave-7kzw krb5kdc[16534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387348, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 18:42:28 dist-test-slave-7kzw krb5kdc[16534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:28 dist-test-slave-7kzw krb5kdc[16534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:28.425508 17127 heartbeater.cc:503] Master 127.4.228.254:33641 was elected leader, sending a full tablet report...
I20260810 18:42:29.037360 17333 heartbeater.cc:503] Master 127.4.228.254:33641 was elected leader, sending a full tablet report...
W20260810 18:42:29.269675 17351 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.166s user 0.010s sys 0.011s
W20260810 18:42:29.271025 17163 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:28.096386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60472 (local address 127.4.228.254:33641)
0810 18:42:28.096623 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:28.096630 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:28.097114 (+ 484us) server_negotiation.cc:407] Connection header received
0810 18:42:28.097415 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:28.097420 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:28.097494 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:28.097618 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:28.099252 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:28.100002 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:28.101538 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:28.101988 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:28.102125 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:29.270770 (+1168645us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60472: BlockingRecv error: recv got EOF from 127.0.0.1:60472 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 10 18:42:29 dist-test-slave-7kzw krb5kdc[16534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387348, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM
I20260810 18:42:29.299643 17163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:29.276234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60474 (local address 127.4.228.254:33641)
0810 18:42:29.276424 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:29.276429 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:29.276884 (+ 455us) server_negotiation.cc:407] Connection header received
0810 18:42:29.277075 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:29.277080 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:29.277164 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:29.277266 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:29.278984 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.280028 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.281698 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.282225 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.282343 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:29.292988 (+ 10645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:29.292995 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:29.292999 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:29.293049 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:29.296409 (+ 3360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:29.297355 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:29.297361 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:29.297363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:29.297523 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:29.298199 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:29.298203 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:29.298205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:29.298567 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:29.299011 (+ 444us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:29.299146 (+ 135us) server_negotiation.cc:299] Negotiation successful
0810 18:42:29.299348 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 18:42:29.304705 16596 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60474:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:42:29.312574 16596 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:42:29.341818 17374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:29.327432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:35775 (local address 127.0.0.1:59436)
0810 18:42:29.328825 (+ 1393us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:29.328878 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:29.329104 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:29.333016 (+ 3912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:29.333021 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:29.333050 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:42:29.333749 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:29.333766 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.336740 (+ 2974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.336748 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:29.338608 (+ 1860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:29.338624 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.340954 (+ 2330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.340961 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:29.341143 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:29.341174 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 18:42:29.341258 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 18:42:29.341396 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"mutex_wait_us":3,"thread_start_us":2005,"threads_started":1}
I20260810 18:42:29.342072 17376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:29.328203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60806 (local address 127.4.228.195:36567)
0810 18:42:29.331331 (+ 3128us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:29.331340 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:29.331370 (+ 30us) server_negotiation.cc:407] Connection header received
0810 18:42:29.331475 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:29.331484 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:29.331799 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:29.332061 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:42:29.333268 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.335042 (+ 1774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.338221 (+ 3179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.339837 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.339995 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:29.340242 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:29.341512 (+ 1270us) server_negotiation.cc:299] Negotiation successful
0810 18:42:29.341725 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2940,"thread_start_us":2785,"threads_started":1}
I20260810 18:42:29.342381 17373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:29.327579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:36567 (local address 127.0.0.1:60806)
0810 18:42:29.329871 (+ 2292us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:29.329894 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:29.330028 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:29.332419 (+ 2391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:29.332431 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:29.332488 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:42:29.333013 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:29.333027 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.335197 (+ 2170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.335204 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:29.338024 (+ 2820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:29.338038 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.340472 (+ 2434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.340481 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:29.340710 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:29.340785 (+ 75us) client_negotiation.cc:769] Sending connection context
0810 18:42:29.340965 (+ 180us) client_negotiation.cc:240] Negotiation successful
0810 18:42:29.341187 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2134,"mutex_wait_us":27,"thread_start_us":799,"threads_started":1}
I20260810 18:42:29.343071 17378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:29.331426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42526 (local address 127.4.228.194:43603)
0810 18:42:29.333117 (+ 1691us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:29.333129 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:29.334764 (+ 1635us) server_negotiation.cc:407] Connection header received
0810 18:42:29.334885 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:29.334893 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:29.335152 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:29.335389 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:42:29.336184 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.337777 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.339964 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.341597 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.341770 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:29.341980 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:29.342521 (+ 541us) server_negotiation.cc:299] Negotiation successful
0810 18:42:29.342743 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1489,"thread_start_us":1316,"threads_started":1}
I20260810 18:42:29.343083 17375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:29.327738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59436 (local address 127.4.228.193:35775)
0810 18:42:29.330273 (+ 2535us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:29.330283 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:29.330317 (+ 34us) server_negotiation.cc:407] Connection header received
0810 18:42:29.330427 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:29.330436 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:29.330755 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:29.333485 (+ 2730us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:42:29.333966 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.336585 (+ 2619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.338822 (+ 2237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.340343 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.340505 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:29.340788 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:29.342536 (+ 1748us) server_negotiation.cc:299] Negotiation successful
0810 18:42:29.342744 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2355,"thread_start_us":2185,"threads_started":1}
I20260810 18:42:29.343968 17377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:29.331102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:43603 (local address 127.0.0.1:42526)
0810 18:42:29.334321 (+ 3219us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:29.334344 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:29.334508 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:29.335394 (+ 886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:29.335400 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:29.335435 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:42:29.335930 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:29.335943 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.337953 (+ 2010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.337958 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:29.339724 (+ 1766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:29.339741 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.341851 (+ 2110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.341858 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:29.342045 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:29.342077 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 18:42:29.342170 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 18:42:29.342325 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2991,"thread_start_us":2816,"threads_started":1}
I20260810 18:42:29.346693 17223 tablet_service.cc:1467] Processing CreateTablet for tablet 199cbeae28a444439e73093f84502468 (DEFAULT_TABLE table=test-table [id=6a6a444864a5471080a8f7c145c1154b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:42:29.346698 16806 tablet_service.cc:1467] Processing CreateTablet for tablet 199cbeae28a444439e73093f84502468 (DEFAULT_TABLE table=test-table [id=6a6a444864a5471080a8f7c145c1154b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:42:29.348196 17223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 199cbeae28a444439e73093f84502468. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:42:29.348287 16806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 199cbeae28a444439e73093f84502468. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:42:29.349996 17010 tablet_service.cc:1467] Processing CreateTablet for tablet 199cbeae28a444439e73093f84502468 (DEFAULT_TABLE table=test-table [id=6a6a444864a5471080a8f7c145c1154b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:42:29.351681 17010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 199cbeae28a444439e73093f84502468. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:42:29.361173 17381 tablet_bootstrap.cc:492] T 199cbeae28a444439e73093f84502468 P 0d3eb3675e6543c98972fe7cf6078944: Bootstrap starting.
I20260810 18:42:29.364904 17381 tablet_bootstrap.cc:654] T 199cbeae28a444439e73093f84502468 P 0d3eb3675e6543c98972fe7cf6078944: Neither blocks nor log segments found. Creating new log.
I20260810 18:42:29.365935 17383 tablet_bootstrap.cc:492] T 199cbeae28a444439e73093f84502468 P baa8dad45c5f493d92d140dea29ec5f9: Bootstrap starting.
I20260810 18:42:29.366528 17381 log.cc:824] T 199cbeae28a444439e73093f84502468 P 0d3eb3675e6543c98972fe7cf6078944: Log is configured to *not* fsync() on all Append() calls
I20260810 18:42:29.370054 17383 tablet_bootstrap.cc:654] T 199cbeae28a444439e73093f84502468 P baa8dad45c5f493d92d140dea29ec5f9: Neither blocks nor log segments found. Creating new log.
I20260810 18:42:29.370203 17381 tablet_bootstrap.cc:492] T 199cbeae28a444439e73093f84502468 P 0d3eb3675e6543c98972fe7cf6078944: No bootstrap required, opened a new log
I20260810 18:42:29.370579 17381 ts_tablet_manager.cc:1397] T 199cbeae28a444439e73093f84502468 P 0d3eb3675e6543c98972fe7cf6078944: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260810 18:42:29.371024 17386 tablet_bootstrap.cc:492] T 199cbeae28a444439e73093f84502468 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: Bootstrap starting.
I20260810 18:42:29.371786 17383 log.cc:824] T 199cbeae28a444439e73093f84502468 P baa8dad45c5f493d92d140dea29ec5f9: Log is configured to *not* fsync() on all Append() calls
I20260810 18:42:29.374934 17386 tablet_bootstrap.cc:654] T 199cbeae28a444439e73093f84502468 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: Neither blocks nor log segments found. Creating new log.
I20260810 18:42:29.375897 17383 tablet_bootstrap.cc:492] T 199cbeae28a444439e73093f84502468 P baa8dad45c5f493d92d140dea29ec5f9: No bootstrap required, opened a new log
I20260810 18:42:29.376094 17386 log.cc:824] T 199cbeae28a444439e73093f84502468 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: Log is configured to *not* fsync() on all Append() calls
I20260810 18:42:29.376231 17383 ts_tablet_manager.cc:1397] T 199cbeae28a444439e73093f84502468 P baa8dad45c5f493d92d140dea29ec5f9: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.007s
I20260810 18:42:29.379000 17386 tablet_bootstrap.cc:492] T 199cbeae28a444439e73093f84502468 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: No bootstrap required, opened a new log
I20260810 18:42:29.379299 17386 ts_tablet_manager.cc:1397] T 199cbeae28a444439e73093f84502468 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260810 18:42:29.382997 17381 raft_consensus.cc:348] T 199cbeae28a444439e73093f84502468 P 0d3eb3675e6543c98972fe7cf6078944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } }
I20260810 18:42:29.383528 17381 raft_consensus.cc:374] T 199cbeae28a444439e73093f84502468 P 0d3eb3675e6543c98972fe7cf6078944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:42:29.383692 17381 raft_consensus.cc:729] T 199cbeae28a444439e73093f84502468 P 0d3eb3675e6543c98972fe7cf6078944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d3eb3675e6543c98972fe7cf6078944, State: Initialized, Role: FOLLOWER
I20260810 18:42:29.384423 17381 consensus_queue.cc:260] T 199cbeae28a444439e73093f84502468 P 0d3eb3675e6543c98972fe7cf6078944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } }
I20260810 18:42:29.386492 17381 ts_tablet_manager.cc:1428] T 199cbeae28a444439e73093f84502468 P 0d3eb3675e6543c98972fe7cf6078944: Time spent starting tablet: real 0.016s user 0.010s sys 0.004s
I20260810 18:42:29.388757 17383 raft_consensus.cc:348] T 199cbeae28a444439e73093f84502468 P baa8dad45c5f493d92d140dea29ec5f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } }
I20260810 18:42:29.389169 17383 raft_consensus.cc:374] T 199cbeae28a444439e73093f84502468 P baa8dad45c5f493d92d140dea29ec5f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:42:29.389305 17383 raft_consensus.cc:729] T 199cbeae28a444439e73093f84502468 P baa8dad45c5f493d92d140dea29ec5f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baa8dad45c5f493d92d140dea29ec5f9, State: Initialized, Role: FOLLOWER
I20260810 18:42:29.389621 17386 raft_consensus.cc:348] T 199cbeae28a444439e73093f84502468 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } }
I20260810 18:42:29.390123 17386 raft_consensus.cc:374] T 199cbeae28a444439e73093f84502468 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:42:29.389957 17383 consensus_queue.cc:260] T 199cbeae28a444439e73093f84502468 P baa8dad45c5f493d92d140dea29ec5f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } }
I20260810 18:42:29.390268 17386 raft_consensus.cc:729] T 199cbeae28a444439e73093f84502468 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e5373d9feb843cf8ef92ae3e7dbb5bc, State: Initialized, Role: FOLLOWER
I20260810 18:42:29.391156 17386 consensus_queue.cc:260] T 199cbeae28a444439e73093f84502468 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } }
I20260810 18:42:29.392052 17383 ts_tablet_manager.cc:1428] T 199cbeae28a444439e73093f84502468 P baa8dad45c5f493d92d140dea29ec5f9: Time spent starting tablet: real 0.016s user 0.006s sys 0.009s
I20260810 18:42:29.395658 17386 ts_tablet_manager.cc:1428] T 199cbeae28a444439e73093f84502468 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: Time spent starting tablet: real 0.016s user 0.014s sys 0.001s
I20260810 18:42:29.413394 17392 raft_consensus.cc:482] T 199cbeae28a444439e73093f84502468 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:42:29.413789 17392 raft_consensus.cc:504] T 199cbeae28a444439e73093f84502468 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } }
I20260810 18:42:29.416302 17392 leader_election.cc:288] T 199cbeae28a444439e73093f84502468 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers baa8dad45c5f493d92d140dea29ec5f9 (127.4.228.193:35775), 0d3eb3675e6543c98972fe7cf6078944 (127.4.228.195:36567)
W20260810 18:42:29.419833 17128 tablet.cc:2369] T 199cbeae28a444439e73093f84502468 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:42:29.424698 17375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:29.416915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:39685 (local address 127.4.228.193:35775)
0810 18:42:29.417144 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:29.417152 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:29.418260 (+ 1108us) server_negotiation.cc:407] Connection header received
0810 18:42:29.418684 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:29.418691 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:29.418777 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:29.418875 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:42:29.419541 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.420623 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.422823 (+ 2200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.423684 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.423866 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:29.423953 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:29.424276 (+ 323us) server_negotiation.cc:299] Negotiation successful
0810 18:42:29.424424 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 18:42:29.424810 17393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:29.416807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:35775 (local address 127.4.228.194:39685)
0810 18:42:29.418127 (+ 1320us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:29.418152 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:29.418418 (+ 266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:29.418903 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:29.418907 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:29.418931 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:42:29.419361 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:29.419376 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.420787 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.420790 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:29.422635 (+ 1845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:29.422651 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.423915 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.423921 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:29.424095 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:29.424132 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 18:42:29.424224 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 18:42:29.424371 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1160,"mutex_wait_us":3,"thread_start_us":613,"threads_started":1}
I20260810 18:42:29.426076 17376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:29.416830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:49861 (local address 127.4.228.195:36567)
0810 18:42:29.417071 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:29.417080 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:29.418734 (+ 1654us) server_negotiation.cc:407] Connection header received
0810 18:42:29.418861 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:29.418867 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:29.418942 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:29.419024 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:42:29.419896 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.420954 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.423719 (+ 2765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.424872 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.425026 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:29.425099 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:29.425696 (+ 597us) server_negotiation.cc:299] Negotiation successful
0810 18:42:29.425842 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 18:42:29.426369 17395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:29.417522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:36567 (local address 127.4.228.194:49861)
0810 18:42:29.418558 (+ 1036us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:29.418596 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:29.418748 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:29.419183 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:29.419187 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:29.419213 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:42:29.419697 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:29.419709 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.421136 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.421142 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:29.423509 (+ 2367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:29.423525 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.425387 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.425392 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:29.425513 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:29.425536 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 18:42:29.425672 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 18:42:29.425823 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":843,"mutex_wait_us":26,"thread_start_us":678,"threads_started":1}
I20260810 18:42:29.426235 16840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "199cbeae28a444439e73093f84502468" candidate_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa8dad45c5f493d92d140dea29ec5f9" is_pre_election: true
I20260810 18:42:29.427081 16840 raft_consensus.cc:2454] T 199cbeae28a444439e73093f84502468 P baa8dad45c5f493d92d140dea29ec5f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e5373d9feb843cf8ef92ae3e7dbb5bc in term 0.
I20260810 18:42:29.427610 17255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "199cbeae28a444439e73093f84502468" candidate_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d3eb3675e6543c98972fe7cf6078944" is_pre_election: true
I20260810 18:42:29.427887 16939 leader_election.cc:302] T 199cbeae28a444439e73093f84502468 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e5373d9feb843cf8ef92ae3e7dbb5bc, baa8dad45c5f493d92d140dea29ec5f9; no voters:
I20260810 18:42:29.428141 17255 raft_consensus.cc:2454] T 199cbeae28a444439e73093f84502468 P 0d3eb3675e6543c98972fe7cf6078944 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e5373d9feb843cf8ef92ae3e7dbb5bc in term 0.
I20260810 18:42:29.428376 17392 raft_consensus.cc:2781] T 199cbeae28a444439e73093f84502468 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:42:29.428531 17392 raft_consensus.cc:482] T 199cbeae28a444439e73093f84502468 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:42:29.428668 17392 raft_consensus.cc:3037] T 199cbeae28a444439e73093f84502468 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:42:29.431412 17392 raft_consensus.cc:504] T 199cbeae28a444439e73093f84502468 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } }
I20260810 18:42:29.432436 17392 leader_election.cc:288] T 199cbeae28a444439e73093f84502468 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [CANDIDATE]: Term 1 election: Requested vote from peers baa8dad45c5f493d92d140dea29ec5f9 (127.4.228.193:35775), 0d3eb3675e6543c98972fe7cf6078944 (127.4.228.195:36567)
I20260810 18:42:29.432873 16840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "199cbeae28a444439e73093f84502468" candidate_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa8dad45c5f493d92d140dea29ec5f9"
I20260810 18:42:29.433162 16840 raft_consensus.cc:3037] T 199cbeae28a444439e73093f84502468 P baa8dad45c5f493d92d140dea29ec5f9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:42:29.433274 17255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "199cbeae28a444439e73093f84502468" candidate_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d3eb3675e6543c98972fe7cf6078944"
I20260810 18:42:29.433507 17255 raft_consensus.cc:3037] T 199cbeae28a444439e73093f84502468 P 0d3eb3675e6543c98972fe7cf6078944 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:42:29.436743 16840 raft_consensus.cc:2454] T 199cbeae28a444439e73093f84502468 P baa8dad45c5f493d92d140dea29ec5f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e5373d9feb843cf8ef92ae3e7dbb5bc in term 1.
I20260810 18:42:29.436779 17255 raft_consensus.cc:2454] T 199cbeae28a444439e73093f84502468 P 0d3eb3675e6543c98972fe7cf6078944 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e5373d9feb843cf8ef92ae3e7dbb5bc in term 1.
I20260810 18:42:29.437283 16940 leader_election.cc:302] T 199cbeae28a444439e73093f84502468 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d3eb3675e6543c98972fe7cf6078944, 6e5373d9feb843cf8ef92ae3e7dbb5bc; no voters:
I20260810 18:42:29.437697 17392 raft_consensus.cc:2781] T 199cbeae28a444439e73093f84502468 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:42:29.438138 17392 raft_consensus.cc:686] T 199cbeae28a444439e73093f84502468 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 1 LEADER]: Becoming Leader. State: Replica: 6e5373d9feb843cf8ef92ae3e7dbb5bc, State: Running, Role: LEADER
I20260810 18:42:29.438759 17392 consensus_queue.cc:237] T 199cbeae28a444439e73093f84502468 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } }
I20260810 18:42:29.443974 16599 catalog_manager.cc:5495] T 199cbeae28a444439e73093f84502468 P 6e5373d9feb843cf8ef92ae3e7dbb5bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e5373d9feb843cf8ef92ae3e7dbb5bc (127.4.228.194). New cstate: current_term: 1 leader_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } health_report { overall_health: HEALTHY } } }
I20260810 18:42:29.471210 17378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:29.465084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42538 (local address 127.4.228.194:43603)
0810 18:42:29.465735 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:29.465740 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:29.465755 (+ 15us) server_negotiation.cc:407] Connection header received
0810 18:42:29.465881 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:29.465886 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:29.466005 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:29.466139 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:42:29.466612 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.467343 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.468727 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.469139 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.469268 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:29.469816 (+ 548us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:42:29.470434 (+ 618us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:42:29.470651 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:29.470798 (+ 147us) server_negotiation.cc:299] Negotiation successful
0810 18:42:29.470877 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514}
I20260810 18:42:29.480997 16840 raft_consensus.cc:1261] T 199cbeae28a444439e73093f84502468 P baa8dad45c5f493d92d140dea29ec5f9 [term 1 FOLLOWER]: Refusing update from remote peer 6e5373d9feb843cf8ef92ae3e7dbb5bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:42:29.480998 17255 raft_consensus.cc:1261] T 199cbeae28a444439e73093f84502468 P 0d3eb3675e6543c98972fe7cf6078944 [term 1 FOLLOWER]: Refusing update from remote peer 6e5373d9feb843cf8ef92ae3e7dbb5bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:42:29.482164 17396 consensus_queue.cc:1035] T 199cbeae28a444439e73093f84502468 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:42:29.482611 17392 consensus_queue.cc:1035] T 199cbeae28a444439e73093f84502468 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:42:29.496243 17400 mvcc.cc:205] Tried to move back new op lower bound from 7317042583462166528 to 7317042583305506816. Current Snapshot: MvccSnapshot[applied={T|T < 7317042583462166528}]
I20260810 18:42:29.500721 17401 mvcc.cc:205] Tried to move back new op lower bound from 7317042583462166528 to 7317042583305506816. Current Snapshot: MvccSnapshot[applied={T|T < 7317042583462166528}]
I20260810 18:42:29.516772 16599 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60474:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:42:29.517167 16599 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60474:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:42:29.522418 16599 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8cfbe048354045c5b31fc749e0cfe06e: Sending DeleteTablet for 3 replicas of tablet 199cbeae28a444439e73093f84502468
I20260810 18:42:29.524326 17223 tablet_service.cc:1514] Processing DeleteTablet for tablet 199cbeae28a444439e73093f84502468 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:42:29 UTC) from {username='oryx', principal='oryx/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:60806
I20260810 18:42:29.524475 17010 tablet_service.cc:1514] Processing DeleteTablet for tablet 199cbeae28a444439e73093f84502468 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:42:29 UTC) from {username='oryx', principal='oryx/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:42526
I20260810 18:42:29.524500 16806 tablet_service.cc:1514] Processing DeleteTablet for tablet 199cbeae28a444439e73093f84502468 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:42:29 UTC) from {username='oryx', principal='oryx/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:59436
I20260810 18:42:29.525843 16596 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60474:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:42:29.527103 16596 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:42:29.534883 17415 tablet_replica.cc:330] T 199cbeae28a444439e73093f84502468 P 0d3eb3675e6543c98972fe7cf6078944: stopping tablet replica
I20260810 18:42:29.535890 17416 tablet_replica.cc:330] T 199cbeae28a444439e73093f84502468 P baa8dad45c5f493d92d140dea29ec5f9: stopping tablet replica
I20260810 18:42:29.535902 17418 tablet_replica.cc:330] T 199cbeae28a444439e73093f84502468 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: stopping tablet replica
I20260810 18:42:29.536422 17418 raft_consensus.cc:2229] T 199cbeae28a444439e73093f84502468 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:42:29.536500 17416 raft_consensus.cc:2229] T 199cbeae28a444439e73093f84502468 P baa8dad45c5f493d92d140dea29ec5f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:42:29.537127 17416 raft_consensus.cc:2258] T 199cbeae28a444439e73093f84502468 P baa8dad45c5f493d92d140dea29ec5f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:42:29.537434 17418 raft_consensus.cc:2258] T 199cbeae28a444439e73093f84502468 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:42:29.539556 17010 tablet_service.cc:1467] Processing CreateTablet for tablet 3ac47f132ad44547988828be15f8a984 (DEFAULT_TABLE table=test-table [id=6da2bb191fdd450a9a522c2072ab2356]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:42:29.539850 17223 tablet_service.cc:1467] Processing CreateTablet for tablet 3ac47f132ad44547988828be15f8a984 (DEFAULT_TABLE table=test-table [id=6da2bb191fdd450a9a522c2072ab2356]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:42:29.540078 17010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ac47f132ad44547988828be15f8a984. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:42:29.540057 17418 ts_tablet_manager.cc:1905] T 199cbeae28a444439e73093f84502468 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:42:29.540472 17223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ac47f132ad44547988828be15f8a984. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:42:29.541013 16806 tablet_service.cc:1467] Processing CreateTablet for tablet 3ac47f132ad44547988828be15f8a984 (DEFAULT_TABLE table=test-table [id=6da2bb191fdd450a9a522c2072ab2356]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:42:29.541502 16806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ac47f132ad44547988828be15f8a984. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:42:29.547117 17416 ts_tablet_manager.cc:1905] T 199cbeae28a444439e73093f84502468 P baa8dad45c5f493d92d140dea29ec5f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:42:29.547827 17418 ts_tablet_manager.cc:1918] T 199cbeae28a444439e73093f84502468 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:42:29.548089 17418 log.cc:1196] T 199cbeae28a444439e73093f84502468 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-1/wal/wals/199cbeae28a444439e73093f84502468
I20260810 18:42:29.548769 17418 ts_tablet_manager.cc:1939] T 199cbeae28a444439e73093f84502468 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: Deleting consensus metadata
I20260810 18:42:29.551113 16574 catalog_manager.cc:4841] TS 6e5373d9feb843cf8ef92ae3e7dbb5bc (127.4.228.194:43603): tablet 199cbeae28a444439e73093f84502468 (table test-table [id=6a6a444864a5471080a8f7c145c1154b]) successfully deleted
I20260810 18:42:29.535518 17415 raft_consensus.cc:2229] T 199cbeae28a444439e73093f84502468 P 0d3eb3675e6543c98972fe7cf6078944 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:42:29.552469 17386 tablet_bootstrap.cc:492] T 3ac47f132ad44547988828be15f8a984 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: Bootstrap starting.
I20260810 18:42:29.553996 17383 tablet_bootstrap.cc:492] T 3ac47f132ad44547988828be15f8a984 P baa8dad45c5f493d92d140dea29ec5f9: Bootstrap starting.
I20260810 18:42:29.554569 17386 tablet_bootstrap.cc:654] T 3ac47f132ad44547988828be15f8a984 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: Neither blocks nor log segments found. Creating new log.
I20260810 18:42:29.554723 17415 raft_consensus.cc:2258] T 199cbeae28a444439e73093f84502468 P 0d3eb3675e6543c98972fe7cf6078944 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:42:29.555891 17383 tablet_bootstrap.cc:654] T 3ac47f132ad44547988828be15f8a984 P baa8dad45c5f493d92d140dea29ec5f9: Neither blocks nor log segments found. Creating new log.
I20260810 18:42:29.556963 17381 tablet_bootstrap.cc:492] T 3ac47f132ad44547988828be15f8a984 P 0d3eb3675e6543c98972fe7cf6078944: Bootstrap starting.
I20260810 18:42:29.558926 17381 tablet_bootstrap.cc:654] T 3ac47f132ad44547988828be15f8a984 P 0d3eb3675e6543c98972fe7cf6078944: Neither blocks nor log segments found. Creating new log.
I20260810 18:42:29.559226 17415 ts_tablet_manager.cc:1905] T 199cbeae28a444439e73093f84502468 P 0d3eb3675e6543c98972fe7cf6078944: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:42:29.566603 17386 tablet_bootstrap.cc:492] T 3ac47f132ad44547988828be15f8a984 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: No bootstrap required, opened a new log
I20260810 18:42:29.566835 17386 ts_tablet_manager.cc:1397] T 3ac47f132ad44547988828be15f8a984 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.008s
I20260810 18:42:29.567852 17386 raft_consensus.cc:348] T 3ac47f132ad44547988828be15f8a984 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } }
I20260810 18:42:29.568131 17386 raft_consensus.cc:374] T 3ac47f132ad44547988828be15f8a984 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:42:29.568228 17386 raft_consensus.cc:729] T 3ac47f132ad44547988828be15f8a984 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e5373d9feb843cf8ef92ae3e7dbb5bc, State: Initialized, Role: FOLLOWER
I20260810 18:42:29.568588 17386 consensus_queue.cc:260] T 3ac47f132ad44547988828be15f8a984 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } }
I20260810 18:42:29.570254 17386 ts_tablet_manager.cc:1428] T 3ac47f132ad44547988828be15f8a984 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 18:42:29.573321 17383 tablet_bootstrap.cc:492] T 3ac47f132ad44547988828be15f8a984 P baa8dad45c5f493d92d140dea29ec5f9: No bootstrap required, opened a new log
I20260810 18:42:29.573552 17383 ts_tablet_manager.cc:1397] T 3ac47f132ad44547988828be15f8a984 P baa8dad45c5f493d92d140dea29ec5f9: Time spent bootstrapping tablet: real 0.020s user 0.009s sys 0.000s
I20260810 18:42:29.574704 17383 raft_consensus.cc:348] T 3ac47f132ad44547988828be15f8a984 P baa8dad45c5f493d92d140dea29ec5f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } }
I20260810 18:42:29.574995 17383 raft_consensus.cc:374] T 3ac47f132ad44547988828be15f8a984 P baa8dad45c5f493d92d140dea29ec5f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:42:29.575125 17383 raft_consensus.cc:729] T 3ac47f132ad44547988828be15f8a984 P baa8dad45c5f493d92d140dea29ec5f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baa8dad45c5f493d92d140dea29ec5f9, State: Initialized, Role: FOLLOWER
I20260810 18:42:29.575474 17383 consensus_queue.cc:260] T 3ac47f132ad44547988828be15f8a984 P baa8dad45c5f493d92d140dea29ec5f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } }
I20260810 18:42:29.577234 17381 tablet_bootstrap.cc:492] T 3ac47f132ad44547988828be15f8a984 P 0d3eb3675e6543c98972fe7cf6078944: No bootstrap required, opened a new log
I20260810 18:42:29.577448 17381 ts_tablet_manager.cc:1397] T 3ac47f132ad44547988828be15f8a984 P 0d3eb3675e6543c98972fe7cf6078944: Time spent bootstrapping tablet: real 0.021s user 0.006s sys 0.009s
I20260810 18:42:29.578866 17383 ts_tablet_manager.cc:1428] T 3ac47f132ad44547988828be15f8a984 P baa8dad45c5f493d92d140dea29ec5f9: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260810 18:42:29.579325 17381 raft_consensus.cc:348] T 3ac47f132ad44547988828be15f8a984 P 0d3eb3675e6543c98972fe7cf6078944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } }
I20260810 18:42:29.579614 17381 raft_consensus.cc:374] T 3ac47f132ad44547988828be15f8a984 P 0d3eb3675e6543c98972fe7cf6078944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:42:29.579821 17381 raft_consensus.cc:729] T 3ac47f132ad44547988828be15f8a984 P 0d3eb3675e6543c98972fe7cf6078944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d3eb3675e6543c98972fe7cf6078944, State: Initialized, Role: FOLLOWER
I20260810 18:42:29.580189 17381 consensus_queue.cc:260] T 3ac47f132ad44547988828be15f8a984 P 0d3eb3675e6543c98972fe7cf6078944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } }
I20260810 18:42:29.580753 17415 ts_tablet_manager.cc:1918] T 199cbeae28a444439e73093f84502468 P 0d3eb3675e6543c98972fe7cf6078944: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:42:29.580966 17415 log.cc:1196] T 199cbeae28a444439e73093f84502468 P 0d3eb3675e6543c98972fe7cf6078944: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-2/wal/wals/199cbeae28a444439e73093f84502468
I20260810 18:42:29.581212 17381 ts_tablet_manager.cc:1428] T 3ac47f132ad44547988828be15f8a984 P 0d3eb3675e6543c98972fe7cf6078944: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260810 18:42:29.581830 17415 ts_tablet_manager.cc:1939] T 199cbeae28a444439e73093f84502468 P 0d3eb3675e6543c98972fe7cf6078944: Deleting consensus metadata
I20260810 18:42:29.583853 16575 catalog_manager.cc:4841] TS 0d3eb3675e6543c98972fe7cf6078944 (127.4.228.195:36567): tablet 199cbeae28a444439e73093f84502468 (table test-table [id=6a6a444864a5471080a8f7c145c1154b]) successfully deleted
I20260810 18:42:29.584589 17416 ts_tablet_manager.cc:1918] T 199cbeae28a444439e73093f84502468 P baa8dad45c5f493d92d140dea29ec5f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:42:29.584789 17416 log.cc:1196] T 199cbeae28a444439e73093f84502468 P baa8dad45c5f493d92d140dea29ec5f9: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-0/wal/wals/199cbeae28a444439e73093f84502468
I20260810 18:42:29.585486 17416 ts_tablet_manager.cc:1939] T 199cbeae28a444439e73093f84502468 P baa8dad45c5f493d92d140dea29ec5f9: Deleting consensus metadata
I20260810 18:42:29.587359 16574 catalog_manager.cc:4841] TS baa8dad45c5f493d92d140dea29ec5f9 (127.4.228.193:35775): tablet 199cbeae28a444439e73093f84502468 (table test-table [id=6a6a444864a5471080a8f7c145c1154b]) successfully deleted
I20260810 18:42:29.604384 17389 raft_consensus.cc:482] T 3ac47f132ad44547988828be15f8a984 P 0d3eb3675e6543c98972fe7cf6078944 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:42:29.604725 17389 raft_consensus.cc:504] T 3ac47f132ad44547988828be15f8a984 P 0d3eb3675e6543c98972fe7cf6078944 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } }
I20260810 18:42:29.606722 17389 leader_election.cc:288] T 3ac47f132ad44547988828be15f8a984 P 0d3eb3675e6543c98972fe7cf6078944 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers baa8dad45c5f493d92d140dea29ec5f9 (127.4.228.193:35775), 6e5373d9feb843cf8ef92ae3e7dbb5bc (127.4.228.194:43603)
I20260810 18:42:29.613472 17391 raft_consensus.cc:482] T 3ac47f132ad44547988828be15f8a984 P baa8dad45c5f493d92d140dea29ec5f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:42:29.613768 17391 raft_consensus.cc:504] T 3ac47f132ad44547988828be15f8a984 P baa8dad45c5f493d92d140dea29ec5f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } }
I20260810 18:42:29.616081 17391 leader_election.cc:288] T 3ac47f132ad44547988828be15f8a984 P baa8dad45c5f493d92d140dea29ec5f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e5373d9feb843cf8ef92ae3e7dbb5bc (127.4.228.194:43603), 0d3eb3675e6543c98972fe7cf6078944 (127.4.228.195:36567)
I20260810 18:42:29.617697 17378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:29.608015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:43871 (local address 127.4.228.194:43603)
0810 18:42:29.608235 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:29.608242 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:29.609010 (+ 768us) server_negotiation.cc:407] Connection header received
0810 18:42:29.609356 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:29.609362 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:29.609449 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:29.609573 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:42:29.610451 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.611654 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.614675 (+ 3021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.616061 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.616259 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:29.616361 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:29.617095 (+ 734us) server_negotiation.cc:299] Negotiation successful
0810 18:42:29.617294 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 18:42:29.618191 17375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:29.607134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:45905 (local address 127.4.228.193:35775)
0810 18:42:29.607376 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:29.607386 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:29.607821 (+ 435us) server_negotiation.cc:407] Connection header received
0810 18:42:29.608207 (+ 386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:29.608218 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:29.608340 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:29.608434 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:42:29.610378 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.611313 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.615425 (+ 4112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.616841 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.616991 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:29.617085 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:29.617763 (+ 678us) server_negotiation.cc:299] Negotiation successful
0810 18:42:29.617908 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260810 18:42:29.618552 17427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:29.607874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:43603 (local address 127.4.228.195:43871)
0810 18:42:29.608836 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:29.608863 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:29.609183 (+ 320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:29.609785 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:29.609791 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:29.609820 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:42:29.610267 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:29.610282 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.611817 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.611823 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:29.614381 (+ 2558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:29.614399 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.616603 (+ 2204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.616610 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:29.616845 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:29.616890 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 18:42:29.617887 (+ 997us) client_negotiation.cc:240] Negotiation successful
0810 18:42:29.618047 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":768,"mutex_wait_us":22,"thread_start_us":536,"threads_started":1}
I20260810 18:42:29.619611 17054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ac47f132ad44547988828be15f8a984" candidate_uuid: "0d3eb3675e6543c98972fe7cf6078944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" is_pre_election: true
I20260810 18:42:29.620141 17054 raft_consensus.cc:2454] T 3ac47f132ad44547988828be15f8a984 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d3eb3675e6543c98972fe7cf6078944 in term 0.
I20260810 18:42:29.620684 17426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:29.606974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:35775 (local address 127.4.228.195:45905)
0810 18:42:29.607697 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:29.607724 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:29.607965 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:29.608740 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:29.608744 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:29.608776 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:42:29.609915 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:29.609935 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.611490 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.611494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:29.615222 (+ 3728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:29.615239 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.617380 (+ 2141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.617387 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:29.617564 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:29.617600 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 18:42:29.618779 (+ 1179us) client_negotiation.cc:240] Negotiation successful
0810 18:42:29.618936 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"mutex_wait_us":130,"thread_start_us":377,"threads_started":1}
I20260810 18:42:29.620988 17152 leader_election.cc:302] T 3ac47f132ad44547988828be15f8a984 P 0d3eb3675e6543c98972fe7cf6078944 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d3eb3675e6543c98972fe7cf6078944, 6e5373d9feb843cf8ef92ae3e7dbb5bc; no voters:
I20260810 18:42:29.621683 17389 raft_consensus.cc:2781] T 3ac47f132ad44547988828be15f8a984 P 0d3eb3675e6543c98972fe7cf6078944 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:42:29.621868 17389 raft_consensus.cc:482] T 3ac47f132ad44547988828be15f8a984 P 0d3eb3675e6543c98972fe7cf6078944 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:42:29.622007 17389 raft_consensus.cc:3037] T 3ac47f132ad44547988828be15f8a984 P 0d3eb3675e6543c98972fe7cf6078944 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:42:29.626519 17389 raft_consensus.cc:504] T 3ac47f132ad44547988828be15f8a984 P 0d3eb3675e6543c98972fe7cf6078944 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } }
I20260810 18:42:29.628521 16840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ac47f132ad44547988828be15f8a984" candidate_uuid: "0d3eb3675e6543c98972fe7cf6078944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa8dad45c5f493d92d140dea29ec5f9" is_pre_election: true
I20260810 18:42:29.628521 17054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ac47f132ad44547988828be15f8a984" candidate_uuid: "0d3eb3675e6543c98972fe7cf6078944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc"
I20260810 18:42:29.628820 17054 raft_consensus.cc:3037] T 3ac47f132ad44547988828be15f8a984 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:42:29.628820 16840 raft_consensus.cc:2454] T 3ac47f132ad44547988828be15f8a984 P baa8dad45c5f493d92d140dea29ec5f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d3eb3675e6543c98972fe7cf6078944 in term 0.
I20260810 18:42:29.630002 16840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ac47f132ad44547988828be15f8a984" candidate_uuid: "0d3eb3675e6543c98972fe7cf6078944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa8dad45c5f493d92d140dea29ec5f9"
I20260810 18:42:29.630224 16840 raft_consensus.cc:3037] T 3ac47f132ad44547988828be15f8a984 P baa8dad45c5f493d92d140dea29ec5f9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:42:29.631635 17378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:29.618015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:47977 (local address 127.4.228.194:43603)
0810 18:42:29.621194 (+ 3179us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:29.621199 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:29.621899 (+ 700us) server_negotiation.cc:407] Connection header received
0810 18:42:29.622799 (+ 900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:29.622806 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:29.622882 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:29.622949 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:42:29.623742 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.624878 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.628533 (+ 3655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.630023 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.630192 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:29.630272 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:29.631119 (+ 847us) server_negotiation.cc:299] Negotiation successful
0810 18:42:29.631299 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3014}
I20260810 18:42:29.632491 17429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:29.616703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:43603 (local address 127.4.228.193:47977)
0810 18:42:29.621722 (+ 5019us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:29.621743 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:29.622624 (+ 881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:29.623176 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:29.623181 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:29.623210 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:42:29.623560 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:29.623572 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.625084 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.625091 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:29.628348 (+ 3257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:29.628363 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.630664 (+ 2301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.630672 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:29.630890 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:29.630931 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 18:42:29.631958 (+ 1027us) client_negotiation.cc:240] Negotiation successful
0810 18:42:29.632123 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4876,"thread_start_us":10483,"threads_started":1}
I20260810 18:42:29.632884 17054 raft_consensus.cc:2454] T 3ac47f132ad44547988828be15f8a984 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d3eb3675e6543c98972fe7cf6078944 in term 1.
I20260810 18:42:29.633685 16840 raft_consensus.cc:2454] T 3ac47f132ad44547988828be15f8a984 P baa8dad45c5f493d92d140dea29ec5f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d3eb3675e6543c98972fe7cf6078944 in term 1.
I20260810 18:42:29.633644 17054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ac47f132ad44547988828be15f8a984" candidate_uuid: "baa8dad45c5f493d92d140dea29ec5f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" is_pre_election: true
I20260810 18:42:29.634006 17054 raft_consensus.cc:2379] T 3ac47f132ad44547988828be15f8a984 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate baa8dad45c5f493d92d140dea29ec5f9 in current term 1: Already voted for candidate 0d3eb3675e6543c98972fe7cf6078944 in this term.
I20260810 18:42:29.634078 17152 leader_election.cc:302] T 3ac47f132ad44547988828be15f8a984 P 0d3eb3675e6543c98972fe7cf6078944 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d3eb3675e6543c98972fe7cf6078944, 6e5373d9feb843cf8ef92ae3e7dbb5bc; no voters:
I20260810 18:42:29.634652 17389 leader_election.cc:288] T 3ac47f132ad44547988828be15f8a984 P 0d3eb3675e6543c98972fe7cf6078944 [CANDIDATE]: Term 1 election: Requested vote from peers baa8dad45c5f493d92d140dea29ec5f9 (127.4.228.193:35775), 6e5373d9feb843cf8ef92ae3e7dbb5bc (127.4.228.194:43603)
I20260810 18:42:29.634959 17389 raft_consensus.cc:2781] T 3ac47f132ad44547988828be15f8a984 P 0d3eb3675e6543c98972fe7cf6078944 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:42:29.635152 17389 raft_consensus.cc:686] T 3ac47f132ad44547988828be15f8a984 P 0d3eb3675e6543c98972fe7cf6078944 [term 1 LEADER]: Becoming Leader. State: Replica: 0d3eb3675e6543c98972fe7cf6078944, State: Running, Role: LEADER
I20260810 18:42:29.635747 17389 consensus_queue.cc:237] T 3ac47f132ad44547988828be15f8a984 P 0d3eb3675e6543c98972fe7cf6078944 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } }
I20260810 18:42:29.640662 17376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:29.618273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:33029 (local address 127.4.228.195:36567)
0810 18:42:29.618504 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:29.618512 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:29.627599 (+ 9087us) server_negotiation.cc:407] Connection header received
0810 18:42:29.627782 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:29.627789 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:29.627880 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:29.627982 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:42:29.633717 (+ 5735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.634983 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.637709 (+ 2726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.639146 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.639296 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:29.639375 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:29.640187 (+ 812us) server_negotiation.cc:299] Negotiation successful
0810 18:42:29.640348 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 18:42:29.641562 17428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:29.617441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:36567 (local address 127.4.228.193:33029)
0810 18:42:29.627500 (+ 10059us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:29.627522 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:29.627654 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:29.633059 (+ 5405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:29.633075 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:29.633107 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:42:29.633466 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:29.633479 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.635325 (+ 1846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.635331 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:29.637497 (+ 2166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:29.637513 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.639575 (+ 2062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.639582 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:29.639948 (+ 366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:29.639991 (+ 43us) client_negotiation.cc:769] Sending connection context
0810 18:42:29.641014 (+ 1023us) client_negotiation.cc:240] Negotiation successful
0810 18:42:29.641221 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9910,"thread_start_us":3976,"threads_started":1}
I20260810 18:42:29.643075 17255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ac47f132ad44547988828be15f8a984" candidate_uuid: "baa8dad45c5f493d92d140dea29ec5f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d3eb3675e6543c98972fe7cf6078944" is_pre_election: true
I20260810 18:42:29.643862 16729 leader_election.cc:302] T 3ac47f132ad44547988828be15f8a984 P baa8dad45c5f493d92d140dea29ec5f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: baa8dad45c5f493d92d140dea29ec5f9; no voters: 0d3eb3675e6543c98972fe7cf6078944, 6e5373d9feb843cf8ef92ae3e7dbb5bc
I20260810 18:42:29.644618 17391 raft_consensus.cc:2726] T 3ac47f132ad44547988828be15f8a984 P baa8dad45c5f493d92d140dea29ec5f9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 18:42:29.645552 16596 catalog_manager.cc:5495] T 3ac47f132ad44547988828be15f8a984 P 0d3eb3675e6543c98972fe7cf6078944 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d3eb3675e6543c98972fe7cf6078944 (127.4.228.195). New cstate: current_term: 1 leader_uuid: "0d3eb3675e6543c98972fe7cf6078944" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } health_report { overall_health: HEALTHY } } }
Aug 10 18:42:29 dist-test-slave-7kzw krb5kdc[16534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387345, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.254@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM
I20260810 18:42:29.709756 17163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:29.693631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60480 (local address 127.4.228.254:33641)
0810 18:42:29.693919 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:29.693928 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:29.694686 (+ 758us) server_negotiation.cc:407] Connection header received
0810 18:42:29.694924 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:29.694929 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:29.695011 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:29.695119 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:29.696827 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.697673 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.699060 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.699478 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.699614 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:29.703403 (+ 3789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:29.703409 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:29.703414 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:29.703454 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:29.706609 (+ 3155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:29.707391 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:29.707396 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:29.707400 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:29.707523 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:29.708054 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:29.708059 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:29.708063 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:29.708386 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:29.708550 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:29.709005 (+ 455us) server_negotiation.cc:299] Negotiation successful
0810 18:42:29.709389 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":175,"server-negotiator.queue_time_us":123}
I20260810 18:42:29.709789 17440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:29.693659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:33641 (local address 127.0.0.1:60480)
0810 18:42:29.694532 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:29.694550 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:29.694788 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:29.695153 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:29.695157 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:29.696176 (+ 1019us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:29.696490 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:29.696544 (+ 54us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.697830 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.697838 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:29.698908 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:29.698921 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.699687 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.699695 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:29.699848 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:29.700719 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:29.700738 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:29.703198 (+ 2460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:29.706765 (+ 3567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:29.706780 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:29.706790 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:29.707221 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:29.707642 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:29.707649 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:29.707653 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:29.707905 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:29.708527 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:29.708537 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:29.708735 (+ 198us) client_negotiation.cc:769] Sending connection context
0810 18:42:29.708913 (+ 178us) client_negotiation.cc:240] Negotiation successful
0810 18:42:29.709268 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"mutex_wait_us":6,"thread_start_us":504,"threads_started":1}
I20260810 18:42:29.716878 16596 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:60480:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 18:42:29.717993 16596 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:42:29.726251 17010 tablet_service.cc:1467] Processing CreateTablet for tablet 38ff61d29f024cd792d15de7546c3412 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c80cc19ec8234365a53839fd588dfd2d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 18:42:29.726313 17223 tablet_service.cc:1467] Processing CreateTablet for tablet 38ff61d29f024cd792d15de7546c3412 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c80cc19ec8234365a53839fd588dfd2d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 18:42:29.727005 17223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38ff61d29f024cd792d15de7546c3412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:42:29.727022 17010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38ff61d29f024cd792d15de7546c3412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:42:29.727510 16806 tablet_service.cc:1467] Processing CreateTablet for tablet 38ff61d29f024cd792d15de7546c3412 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c80cc19ec8234365a53839fd588dfd2d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 18:42:29.728094 16806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38ff61d29f024cd792d15de7546c3412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:42:29.734975 17381 tablet_bootstrap.cc:492] T 38ff61d29f024cd792d15de7546c3412 P 0d3eb3675e6543c98972fe7cf6078944: Bootstrap starting.
I20260810 18:42:29.736204 17386 tablet_bootstrap.cc:492] T 38ff61d29f024cd792d15de7546c3412 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: Bootstrap starting.
I20260810 18:42:29.736863 17383 tablet_bootstrap.cc:492] T 38ff61d29f024cd792d15de7546c3412 P baa8dad45c5f493d92d140dea29ec5f9: Bootstrap starting.
I20260810 18:42:29.736893 17381 tablet_bootstrap.cc:654] T 38ff61d29f024cd792d15de7546c3412 P 0d3eb3675e6543c98972fe7cf6078944: Neither blocks nor log segments found. Creating new log.
I20260810 18:42:29.738061 17386 tablet_bootstrap.cc:654] T 38ff61d29f024cd792d15de7546c3412 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: Neither blocks nor log segments found. Creating new log.
I20260810 18:42:29.739059 17383 tablet_bootstrap.cc:654] T 38ff61d29f024cd792d15de7546c3412 P baa8dad45c5f493d92d140dea29ec5f9: Neither blocks nor log segments found. Creating new log.
I20260810 18:42:29.747449 17383 tablet_bootstrap.cc:492] T 38ff61d29f024cd792d15de7546c3412 P baa8dad45c5f493d92d140dea29ec5f9: No bootstrap required, opened a new log
I20260810 18:42:29.747700 17383 ts_tablet_manager.cc:1397] T 38ff61d29f024cd792d15de7546c3412 P baa8dad45c5f493d92d140dea29ec5f9: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260810 18:42:29.749215 17386 tablet_bootstrap.cc:492] T 38ff61d29f024cd792d15de7546c3412 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: No bootstrap required, opened a new log
I20260810 18:42:29.749441 17381 tablet_bootstrap.cc:492] T 38ff61d29f024cd792d15de7546c3412 P 0d3eb3675e6543c98972fe7cf6078944: No bootstrap required, opened a new log
I20260810 18:42:29.749611 17386 ts_tablet_manager.cc:1397] T 38ff61d29f024cd792d15de7546c3412 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.007s
I20260810 18:42:29.749651 17381 ts_tablet_manager.cc:1397] T 38ff61d29f024cd792d15de7546c3412 P 0d3eb3675e6543c98972fe7cf6078944: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.008s
I20260810 18:42:29.748631 17383 raft_consensus.cc:348] T 38ff61d29f024cd792d15de7546c3412 P baa8dad45c5f493d92d140dea29ec5f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } }
I20260810 18:42:29.750149 17383 raft_consensus.cc:374] T 38ff61d29f024cd792d15de7546c3412 P baa8dad45c5f493d92d140dea29ec5f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:42:29.750255 17383 raft_consensus.cc:729] T 38ff61d29f024cd792d15de7546c3412 P baa8dad45c5f493d92d140dea29ec5f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baa8dad45c5f493d92d140dea29ec5f9, State: Initialized, Role: FOLLOWER
I20260810 18:42:29.750504 17381 raft_consensus.cc:348] T 38ff61d29f024cd792d15de7546c3412 P 0d3eb3675e6543c98972fe7cf6078944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } }
I20260810 18:42:29.750592 17383 consensus_queue.cc:260] T 38ff61d29f024cd792d15de7546c3412 P baa8dad45c5f493d92d140dea29ec5f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } }
I20260810 18:42:29.750779 17381 raft_consensus.cc:374] T 38ff61d29f024cd792d15de7546c3412 P 0d3eb3675e6543c98972fe7cf6078944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:42:29.750867 17381 raft_consensus.cc:729] T 38ff61d29f024cd792d15de7546c3412 P 0d3eb3675e6543c98972fe7cf6078944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d3eb3675e6543c98972fe7cf6078944, State: Initialized, Role: FOLLOWER
I20260810 18:42:29.751020 17386 raft_consensus.cc:348] T 38ff61d29f024cd792d15de7546c3412 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } }
I20260810 18:42:29.751112 17391 tablet_replica.cc:439] T 38ff61d29f024cd792d15de7546c3412 P baa8dad45c5f493d92d140dea29ec5f9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } } }
I20260810 18:42:29.751165 17381 consensus_queue.cc:260] T 38ff61d29f024cd792d15de7546c3412 P 0d3eb3675e6543c98972fe7cf6078944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } }
I20260810 18:42:29.751426 17391 tablet_replica.cc:442] T 38ff61d29f024cd792d15de7546c3412 P baa8dad45c5f493d92d140dea29ec5f9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:42:29.751811 17381 ts_tablet_manager.cc:1428] T 38ff61d29f024cd792d15de7546c3412 P 0d3eb3675e6543c98972fe7cf6078944: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 18:42:29.752535 17383 ts_tablet_manager.cc:1428] T 38ff61d29f024cd792d15de7546c3412 P baa8dad45c5f493d92d140dea29ec5f9: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260810 18:42:29.752512 17389 tablet_replica.cc:439] T 38ff61d29f024cd792d15de7546c3412 P 0d3eb3675e6543c98972fe7cf6078944: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } } }
I20260810 18:42:29.752820 17389 tablet_replica.cc:442] T 38ff61d29f024cd792d15de7546c3412 P 0d3eb3675e6543c98972fe7cf6078944: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:42:29.753057 17386 raft_consensus.cc:374] T 38ff61d29f024cd792d15de7546c3412 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:42:29.753229 17386 raft_consensus.cc:729] T 38ff61d29f024cd792d15de7546c3412 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e5373d9feb843cf8ef92ae3e7dbb5bc, State: Initialized, Role: FOLLOWER
I20260810 18:42:29.753602 17386 consensus_queue.cc:260] T 38ff61d29f024cd792d15de7546c3412 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } }
I20260810 18:42:29.754233 17396 tablet_replica.cc:439] T 38ff61d29f024cd792d15de7546c3412 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } } }
I20260810 18:42:29.754652 17396 tablet_replica.cc:442] T 38ff61d29f024cd792d15de7546c3412 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:42:29.755302 17386 ts_tablet_manager.cc:1428] T 38ff61d29f024cd792d15de7546c3412 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260810 18:42:29.775911 17396 raft_consensus.cc:482] T 38ff61d29f024cd792d15de7546c3412 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:42:29.776332 17396 raft_consensus.cc:504] T 38ff61d29f024cd792d15de7546c3412 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } }
I20260810 18:42:29.777983 16840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38ff61d29f024cd792d15de7546c3412" candidate_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa8dad45c5f493d92d140dea29ec5f9" is_pre_election: true
I20260810 18:42:29.778285 16840 raft_consensus.cc:2454] T 38ff61d29f024cd792d15de7546c3412 P baa8dad45c5f493d92d140dea29ec5f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e5373d9feb843cf8ef92ae3e7dbb5bc in term 0.
I20260810 18:42:29.778259 17255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38ff61d29f024cd792d15de7546c3412" candidate_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d3eb3675e6543c98972fe7cf6078944" is_pre_election: true
I20260810 18:42:29.778725 17255 raft_consensus.cc:2454] T 38ff61d29f024cd792d15de7546c3412 P 0d3eb3675e6543c98972fe7cf6078944 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e5373d9feb843cf8ef92ae3e7dbb5bc in term 0.
I20260810 18:42:29.778788 17396 leader_election.cc:288] T 38ff61d29f024cd792d15de7546c3412 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers baa8dad45c5f493d92d140dea29ec5f9 (127.4.228.193:35775), 0d3eb3675e6543c98972fe7cf6078944 (127.4.228.195:36567)
I20260810 18:42:29.778790 16939 leader_election.cc:302] T 38ff61d29f024cd792d15de7546c3412 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e5373d9feb843cf8ef92ae3e7dbb5bc, baa8dad45c5f493d92d140dea29ec5f9; no voters:
I20260810 18:42:29.779214 17396 raft_consensus.cc:2781] T 38ff61d29f024cd792d15de7546c3412 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:42:29.779332 17396 raft_consensus.cc:482] T 38ff61d29f024cd792d15de7546c3412 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:42:29.779448 17396 raft_consensus.cc:3037] T 38ff61d29f024cd792d15de7546c3412 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 0 FOLLOWER]: Advancing to term 1
W20260810 18:42:29.781690 17336 tablet.cc:2369] T 38ff61d29f024cd792d15de7546c3412 P 0d3eb3675e6543c98972fe7cf6078944: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:42:29.782727 17396 raft_consensus.cc:504] T 38ff61d29f024cd792d15de7546c3412 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } }
I20260810 18:42:29.783581 17396 leader_election.cc:288] T 38ff61d29f024cd792d15de7546c3412 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [CANDIDATE]: Term 1 election: Requested vote from peers baa8dad45c5f493d92d140dea29ec5f9 (127.4.228.193:35775), 0d3eb3675e6543c98972fe7cf6078944 (127.4.228.195:36567)
I20260810 18:42:29.783871 16840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38ff61d29f024cd792d15de7546c3412" candidate_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa8dad45c5f493d92d140dea29ec5f9"
I20260810 18:42:29.784137 16840 raft_consensus.cc:3037] T 38ff61d29f024cd792d15de7546c3412 P baa8dad45c5f493d92d140dea29ec5f9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:42:29.784142 17255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38ff61d29f024cd792d15de7546c3412" candidate_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d3eb3675e6543c98972fe7cf6078944"
I20260810 18:42:29.784368 17255 raft_consensus.cc:3037] T 38ff61d29f024cd792d15de7546c3412 P 0d3eb3675e6543c98972fe7cf6078944 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:42:29.787338 16840 raft_consensus.cc:2454] T 38ff61d29f024cd792d15de7546c3412 P baa8dad45c5f493d92d140dea29ec5f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e5373d9feb843cf8ef92ae3e7dbb5bc in term 1.
I20260810 18:42:29.787358 17255 raft_consensus.cc:2454] T 38ff61d29f024cd792d15de7546c3412 P 0d3eb3675e6543c98972fe7cf6078944 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e5373d9feb843cf8ef92ae3e7dbb5bc in term 1.
I20260810 18:42:29.787817 16939 leader_election.cc:302] T 38ff61d29f024cd792d15de7546c3412 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e5373d9feb843cf8ef92ae3e7dbb5bc, baa8dad45c5f493d92d140dea29ec5f9; no voters:
I20260810 18:42:29.788255 17396 raft_consensus.cc:2781] T 38ff61d29f024cd792d15de7546c3412 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:42:29.788641 17396 raft_consensus.cc:686] T 38ff61d29f024cd792d15de7546c3412 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 1 LEADER]: Becoming Leader. State: Replica: 6e5373d9feb843cf8ef92ae3e7dbb5bc, State: Running, Role: LEADER
I20260810 18:42:29.788961 17396 consensus_queue.cc:237] T 38ff61d29f024cd792d15de7546c3412 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } }
W20260810 18:42:29.789393 16907 tablet.cc:2369] T 38ff61d29f024cd792d15de7546c3412 P baa8dad45c5f493d92d140dea29ec5f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:42:29.791352 17412 tablet_replica.cc:439] T 38ff61d29f024cd792d15de7546c3412 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: TxnStatusTablet state changed. Reason: New leader 6e5373d9feb843cf8ef92ae3e7dbb5bc. Latest consensus state: current_term: 1 leader_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } } }
I20260810 18:42:29.791782 17412 tablet_replica.cc:442] T 38ff61d29f024cd792d15de7546c3412 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: This TxnStatusTablet replica's current role is: LEADER
I20260810 18:42:29.794070 16596 catalog_manager.cc:5495] T 38ff61d29f024cd792d15de7546c3412 P 6e5373d9feb843cf8ef92ae3e7dbb5bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e5373d9feb843cf8ef92ae3e7dbb5bc (127.4.228.194). New cstate: current_term: 1 leader_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } health_report { overall_health: UNKNOWN } } }
I20260810 18:42:29.797379 17446 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 18:42:29.797724 17446 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 18:42:29.824496 17440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:29.817906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:43603 (local address 127.0.0.1:42554)
0810 18:42:29.818160 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:29.818181 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:29.818326 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:29.818866 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:29.818870 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:29.818881 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:42:29.819053 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:29.819060 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.820503 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.820508 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:29.821793 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:29.821809 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.823173 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.823178 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:29.823352 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:29.823425 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:42:29.823968 (+ 543us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:42:29.824008 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 18:42:29.824083 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 18:42:29.824219 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260810 18:42:29.824561 17378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:29.817951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42554 (local address 127.4.228.194:43603)
0810 18:42:29.818175 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:29.818181 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:29.818286 (+ 105us) server_negotiation.cc:407] Connection header received
0810 18:42:29.818546 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:29.818556 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:29.818654 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:29.818787 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:42:29.819200 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.820328 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.821959 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.822946 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.823091 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:29.823664 (+ 573us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:42:29.823830 (+ 166us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:42:29.823929 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:29.824139 (+ 210us) server_negotiation.cc:299] Negotiation successful
0810 18:42:29.824226 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 18:42:29.829483 17255 raft_consensus.cc:1261] T 38ff61d29f024cd792d15de7546c3412 P 0d3eb3675e6543c98972fe7cf6078944 [term 1 FOLLOWER]: Refusing update from remote peer 6e5373d9feb843cf8ef92ae3e7dbb5bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:42:29.829540 16840 raft_consensus.cc:1261] T 38ff61d29f024cd792d15de7546c3412 P baa8dad45c5f493d92d140dea29ec5f9 [term 1 FOLLOWER]: Refusing update from remote peer 6e5373d9feb843cf8ef92ae3e7dbb5bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:42:29.830237 17412 consensus_queue.cc:1035] T 38ff61d29f024cd792d15de7546c3412 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:42:29.830706 17396 consensus_queue.cc:1035] T 38ff61d29f024cd792d15de7546c3412 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:42:29.834116 17391 tablet_replica.cc:439] T 38ff61d29f024cd792d15de7546c3412 P baa8dad45c5f493d92d140dea29ec5f9: TxnStatusTablet state changed. Reason: New leader 6e5373d9feb843cf8ef92ae3e7dbb5bc. Latest consensus state: current_term: 1 leader_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } } }
I20260810 18:42:29.834458 17391 tablet_replica.cc:442] T 38ff61d29f024cd792d15de7546c3412 P baa8dad45c5f493d92d140dea29ec5f9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:42:29.837173 17396 tablet_replica.cc:439] T 38ff61d29f024cd792d15de7546c3412 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } } }
I20260810 18:42:29.837532 17396 tablet_replica.cc:442] T 38ff61d29f024cd792d15de7546c3412 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: This TxnStatusTablet replica's current role is: LEADER
I20260810 18:42:29.839471 17391 tablet_replica.cc:439] T 38ff61d29f024cd792d15de7546c3412 P baa8dad45c5f493d92d140dea29ec5f9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } } }
I20260810 18:42:29.839759 17391 tablet_replica.cc:442] T 38ff61d29f024cd792d15de7546c3412 P baa8dad45c5f493d92d140dea29ec5f9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:42:29.847525 17389 tablet_replica.cc:439] T 38ff61d29f024cd792d15de7546c3412 P 0d3eb3675e6543c98972fe7cf6078944: TxnStatusTablet state changed. Reason: New leader 6e5373d9feb843cf8ef92ae3e7dbb5bc. Latest consensus state: current_term: 1 leader_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } } }
I20260810 18:42:29.847934 17389 tablet_replica.cc:442] T 38ff61d29f024cd792d15de7546c3412 P 0d3eb3675e6543c98972fe7cf6078944: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:42:29.850238 17412 tablet_replica.cc:439] T 38ff61d29f024cd792d15de7546c3412 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } } }
I20260810 18:42:29.850621 17412 tablet_replica.cc:442] T 38ff61d29f024cd792d15de7546c3412 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: This TxnStatusTablet replica's current role is: LEADER
I20260810 18:42:29.855141 17389 tablet_replica.cc:439] T 38ff61d29f024cd792d15de7546c3412 P 0d3eb3675e6543c98972fe7cf6078944: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } } }
I20260810 18:42:29.855440 17389 tablet_replica.cc:442] T 38ff61d29f024cd792d15de7546c3412 P 0d3eb3675e6543c98972fe7cf6078944: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:42:29.857378 17376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:29.849196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60816 (local address 127.4.228.195:36567)
0810 18:42:29.849402 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:29.849409 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:29.849429 (+ 20us) server_negotiation.cc:407] Connection header received
0810 18:42:29.849534 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:29.849542 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:29.849665 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:29.849852 (+ 187us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:42:29.850385 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.851514 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.854041 (+ 2527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.854812 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.854955 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:29.855682 (+ 727us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:42:29.856507 (+ 825us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:42:29.856669 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:29.856923 (+ 254us) server_negotiation.cc:299] Negotiation successful
0810 18:42:29.857030 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 18:42:29.879573 17378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:29.871271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:57247 (local address 127.4.228.194:43603)
0810 18:42:29.871522 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:29.871532 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:29.872294 (+ 762us) server_negotiation.cc:407] Connection header received
0810 18:42:29.872706 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:29.872711 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:29.872785 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:29.872870 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:42:29.873712 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.874660 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.877188 (+ 2528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.877710 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.877839 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:29.878566 (+ 727us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:42:29.878712 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:42:29.878790 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:29.879075 (+ 285us) server_negotiation.cc:299] Negotiation successful
0810 18:42:29.879176 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 18:42:29.879953 17455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:29.871262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:43603 (local address 127.4.228.195:57247)
0810 18:42:29.872097 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:29.872121 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:29.872401 (+ 280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:29.872942 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:29.872948 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:29.872961 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:42:29.873336 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:29.873351 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.874912 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.874918 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:29.876923 (+ 2005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:29.876946 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.877948 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.877957 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:29.878169 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:29.878273 (+ 104us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:42:29.878860 (+ 587us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:42:29.878877 (+ 17us) client_negotiation.cc:769] Sending connection context
0810 18:42:29.878997 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 18:42:29.879278 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":447,"threads_started":1}
I20260810 18:42:29.892385 17054 raft_consensus.cc:1261] T 3ac47f132ad44547988828be15f8a984 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 1 FOLLOWER]: Refusing update from remote peer 0d3eb3675e6543c98972fe7cf6078944: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:42:29.892354 16840 raft_consensus.cc:1261] T 3ac47f132ad44547988828be15f8a984 P baa8dad45c5f493d92d140dea29ec5f9 [term 1 FOLLOWER]: Refusing update from remote peer 0d3eb3675e6543c98972fe7cf6078944: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:42:29.893167 17389 consensus_queue.cc:1035] T 3ac47f132ad44547988828be15f8a984 P 0d3eb3675e6543c98972fe7cf6078944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:42:29.893631 17430 consensus_queue.cc:1035] T 3ac47f132ad44547988828be15f8a984 P 0d3eb3675e6543c98972fe7cf6078944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:42:29.902832 17403 mvcc.cc:205] Tried to move back new op lower bound from 7317042585149812736 to 7317042584114630656. Current Snapshot: MvccSnapshot[applied={T|T < 7317042585149812736}]
I20260810 18:42:29.957491 17376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:29.944402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:44031 (local address 127.4.228.195:36567)
0810 18:42:29.944698 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:29.944708 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:29.950259 (+ 5551us) server_negotiation.cc:407] Connection header received
0810 18:42:29.950698 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:29.950704 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:29.950812 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:29.950889 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:42:29.951919 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.952678 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.954968 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.955748 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.955899 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:29.956541 (+ 642us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:42:29.956702 (+ 161us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:42:29.956814 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:29.957041 (+ 227us) server_negotiation.cc:299] Negotiation successful
0810 18:42:29.957128 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 18:42:29.957607 17465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:29.944244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:36567 (local address 127.4.228.194:44031)
0810 18:42:29.950120 (+ 5876us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:29.950156 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:29.950450 (+ 294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:29.951118 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:29.951122 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:29.951136 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:42:29.951634 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:29.951662 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.952837 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.952844 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:29.954770 (+ 1926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:29.954789 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:29.955979 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:29.955987 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:29.956206 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:29.956302 (+ 96us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:42:29.956872 (+ 570us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:42:29.956884 (+ 12us) client_negotiation.cc:769] Sending connection context
0810 18:42:29.957010 (+ 126us) client_negotiation.cc:240] Negotiation successful
0810 18:42:29.957168 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5633,"thread_start_us":5473,"threads_started":1}
I20260810 18:42:30.015189 16596 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60474:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:42:30.015488 16596 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60474:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:42:30.019469 16596 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8cfbe048354045c5b31fc749e0cfe06e: Sending DeleteTablet for 3 replicas of tablet 3ac47f132ad44547988828be15f8a984
I20260810 18:42:30.021080 16806 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ac47f132ad44547988828be15f8a984 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:42:30 UTC) from {username='oryx', principal='oryx/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:59436
I20260810 18:42:30.021443 17416 tablet_replica.cc:330] T 3ac47f132ad44547988828be15f8a984 P baa8dad45c5f493d92d140dea29ec5f9: stopping tablet replica
I20260810 18:42:30.021534 17223 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ac47f132ad44547988828be15f8a984 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:42:30 UTC) from {username='oryx', principal='oryx/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:60806
I20260810 18:42:30.021725 17416 raft_consensus.cc:2229] T 3ac47f132ad44547988828be15f8a984 P baa8dad45c5f493d92d140dea29ec5f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:42:30.021893 17415 tablet_replica.cc:330] T 3ac47f132ad44547988828be15f8a984 P 0d3eb3675e6543c98972fe7cf6078944: stopping tablet replica
I20260810 18:42:30.021981 17416 raft_consensus.cc:2258] T 3ac47f132ad44547988828be15f8a984 P baa8dad45c5f493d92d140dea29ec5f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:42:30.022122 17415 raft_consensus.cc:2229] T 3ac47f132ad44547988828be15f8a984 P 0d3eb3675e6543c98972fe7cf6078944 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:42:30.022872 17415 raft_consensus.cc:2258] T 3ac47f132ad44547988828be15f8a984 P 0d3eb3675e6543c98972fe7cf6078944 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:42:30.023130 17010 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ac47f132ad44547988828be15f8a984 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:42:30 UTC) from {username='oryx', principal='oryx/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:42526
I20260810 18:42:30.023622 17418 tablet_replica.cc:330] T 3ac47f132ad44547988828be15f8a984 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: stopping tablet replica
I20260810 18:42:30.023748 17416 ts_tablet_manager.cc:1905] T 3ac47f132ad44547988828be15f8a984 P baa8dad45c5f493d92d140dea29ec5f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:42:30.023869 17418 raft_consensus.cc:2229] T 3ac47f132ad44547988828be15f8a984 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:42:30.024102 17418 raft_consensus.cc:2258] T 3ac47f132ad44547988828be15f8a984 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:42:30.025616 17418 ts_tablet_manager.cc:1905] T 3ac47f132ad44547988828be15f8a984 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:42:30.027377 17415 ts_tablet_manager.cc:1905] T 3ac47f132ad44547988828be15f8a984 P 0d3eb3675e6543c98972fe7cf6078944: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:42:30.041024 17416 ts_tablet_manager.cc:1918] T 3ac47f132ad44547988828be15f8a984 P baa8dad45c5f493d92d140dea29ec5f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:42:30.041247 17416 log.cc:1196] T 3ac47f132ad44547988828be15f8a984 P baa8dad45c5f493d92d140dea29ec5f9: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-0/wal/wals/3ac47f132ad44547988828be15f8a984
I20260810 18:42:30.041765 17416 ts_tablet_manager.cc:1939] T 3ac47f132ad44547988828be15f8a984 P baa8dad45c5f493d92d140dea29ec5f9: Deleting consensus metadata
I20260810 18:42:30.043569 16574 catalog_manager.cc:4841] TS baa8dad45c5f493d92d140dea29ec5f9 (127.4.228.193:35775): tablet 3ac47f132ad44547988828be15f8a984 (table test-table [id=6da2bb191fdd450a9a522c2072ab2356]) successfully deleted
I20260810 18:42:30.049789 17415 ts_tablet_manager.cc:1918] T 3ac47f132ad44547988828be15f8a984 P 0d3eb3675e6543c98972fe7cf6078944: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:42:30.053632 17418 ts_tablet_manager.cc:1918] T 3ac47f132ad44547988828be15f8a984 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:42:30.053905 17418 log.cc:1196] T 3ac47f132ad44547988828be15f8a984 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-1/wal/wals/3ac47f132ad44547988828be15f8a984
I20260810 18:42:30.058751 17418 ts_tablet_manager.cc:1939] T 3ac47f132ad44547988828be15f8a984 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: Deleting consensus metadata
I20260810 18:42:30.060456 16574 catalog_manager.cc:4841] TS 6e5373d9feb843cf8ef92ae3e7dbb5bc (127.4.228.194:43603): tablet 3ac47f132ad44547988828be15f8a984 (table test-table [id=6da2bb191fdd450a9a522c2072ab2356]) successfully deleted
I20260810 18:42:30.067409 17415 log.cc:1196] T 3ac47f132ad44547988828be15f8a984 P 0d3eb3675e6543c98972fe7cf6078944: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-2/wal/wals/3ac47f132ad44547988828be15f8a984
I20260810 18:42:30.070925 17415 ts_tablet_manager.cc:1939] T 3ac47f132ad44547988828be15f8a984 P 0d3eb3675e6543c98972fe7cf6078944: Deleting consensus metadata
I20260810 18:42:30.072345 16575 catalog_manager.cc:4841] TS 0d3eb3675e6543c98972fe7cf6078944 (127.4.228.195:36567): tablet 3ac47f132ad44547988828be15f8a984 (table test-table [id=6da2bb191fdd450a9a522c2072ab2356]) successfully deleted
Aug 10 18:42:30 dist-test-slave-7kzw krb5kdc[16534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 18:42:30 dist-test-slave-7kzw krb5kdc[16534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:30 dist-test-slave-7kzw krb5kdc[16534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
W20260810 18:42:30.151991 17163 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:30.139186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60484 (local address 127.4.228.254:33641)
0810 18:42:30.139430 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:30.139440 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:30.140037 (+ 597us) server_negotiation.cc:407] Connection header received
0810 18:42:30.140342 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:30.140346 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:30.140426 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:30.140547 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:30.142292 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:30.143277 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:30.144609 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:30.145043 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:30.145195 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:30.151757 (+ 6562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60484: BlockingRecv error: recv got EOF from 127.0.0.1:60484 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 10 18:42:30 dist-test-slave-7kzw krb5kdc[16534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM
I20260810 18:42:30.176474 17163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:30.157022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60492 (local address 127.4.228.254:33641)
0810 18:42:30.157317 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:30.157329 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:30.159211 (+ 1882us) server_negotiation.cc:407] Connection header received
0810 18:42:30.159494 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:30.159526 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:30.159667 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:30.159904 (+ 237us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:30.161476 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:30.162531 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:30.164000 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:30.164452 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:30.164601 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:30.169444 (+ 4843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:30.169450 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:30.169453 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:30.169498 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:30.172896 (+ 3398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:30.173799 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:30.173806 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:30.173812 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:30.173993 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:30.174537 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:30.174542 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:30.174544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:30.174880 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:30.175218 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:30.175596 (+ 378us) server_negotiation.cc:299] Negotiation successful
0810 18:42:30.175890 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260810 18:42:30.181282 16596 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60492:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:42:30.182336 16596 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:42:30.190809 16806 tablet_service.cc:1467] Processing CreateTablet for tablet 25033b432cb4452096869acc626bc359 (DEFAULT_TABLE table=test-table [id=7495e0fb1e3a4d59a99e87e19ff1860a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:42:30.191226 17010 tablet_service.cc:1467] Processing CreateTablet for tablet 25033b432cb4452096869acc626bc359 (DEFAULT_TABLE table=test-table [id=7495e0fb1e3a4d59a99e87e19ff1860a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:42:30.191397 16806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25033b432cb4452096869acc626bc359. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:42:30.191794 17010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25033b432cb4452096869acc626bc359. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:42:30.195551 17223 tablet_service.cc:1467] Processing CreateTablet for tablet 25033b432cb4452096869acc626bc359 (DEFAULT_TABLE table=test-table [id=7495e0fb1e3a4d59a99e87e19ff1860a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:42:30.196198 17223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25033b432cb4452096869acc626bc359. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:42:30.204385 17383 tablet_bootstrap.cc:492] T 25033b432cb4452096869acc626bc359 P baa8dad45c5f493d92d140dea29ec5f9: Bootstrap starting.
I20260810 18:42:30.206426 17383 tablet_bootstrap.cc:654] T 25033b432cb4452096869acc626bc359 P baa8dad45c5f493d92d140dea29ec5f9: Neither blocks nor log segments found. Creating new log.
I20260810 18:42:30.210556 17381 tablet_bootstrap.cc:492] T 25033b432cb4452096869acc626bc359 P 0d3eb3675e6543c98972fe7cf6078944: Bootstrap starting.
I20260810 18:42:30.213312 17381 tablet_bootstrap.cc:654] T 25033b432cb4452096869acc626bc359 P 0d3eb3675e6543c98972fe7cf6078944: Neither blocks nor log segments found. Creating new log.
I20260810 18:42:30.217084 17386 tablet_bootstrap.cc:492] T 25033b432cb4452096869acc626bc359 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: Bootstrap starting.
I20260810 18:42:30.219825 17386 tablet_bootstrap.cc:654] T 25033b432cb4452096869acc626bc359 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: Neither blocks nor log segments found. Creating new log.
I20260810 18:42:30.223227 17386 tablet_bootstrap.cc:492] T 25033b432cb4452096869acc626bc359 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: No bootstrap required, opened a new log
I20260810 18:42:30.223562 17386 ts_tablet_manager.cc:1397] T 25033b432cb4452096869acc626bc359 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260810 18:42:30.224844 17386 raft_consensus.cc:348] T 25033b432cb4452096869acc626bc359 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } }
I20260810 18:42:30.227317 17386 raft_consensus.cc:374] T 25033b432cb4452096869acc626bc359 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:42:30.227463 17381 tablet_bootstrap.cc:492] T 25033b432cb4452096869acc626bc359 P 0d3eb3675e6543c98972fe7cf6078944: No bootstrap required, opened a new log
I20260810 18:42:30.227568 17386 raft_consensus.cc:729] T 25033b432cb4452096869acc626bc359 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e5373d9feb843cf8ef92ae3e7dbb5bc, State: Initialized, Role: FOLLOWER
I20260810 18:42:30.227669 17381 ts_tablet_manager.cc:1397] T 25033b432cb4452096869acc626bc359 P 0d3eb3675e6543c98972fe7cf6078944: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.010s
I20260810 18:42:30.228040 17386 consensus_queue.cc:260] T 25033b432cb4452096869acc626bc359 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } }
I20260810 18:42:30.228613 17381 raft_consensus.cc:348] T 25033b432cb4452096869acc626bc359 P 0d3eb3675e6543c98972fe7cf6078944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } }
I20260810 18:42:30.229104 17381 raft_consensus.cc:374] T 25033b432cb4452096869acc626bc359 P 0d3eb3675e6543c98972fe7cf6078944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:42:30.229220 17381 raft_consensus.cc:729] T 25033b432cb4452096869acc626bc359 P 0d3eb3675e6543c98972fe7cf6078944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d3eb3675e6543c98972fe7cf6078944, State: Initialized, Role: FOLLOWER
I20260810 18:42:30.229598 17381 consensus_queue.cc:260] T 25033b432cb4452096869acc626bc359 P 0d3eb3675e6543c98972fe7cf6078944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } }
I20260810 18:42:30.231846 17381 ts_tablet_manager.cc:1428] T 25033b432cb4452096869acc626bc359 P 0d3eb3675e6543c98972fe7cf6078944: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260810 18:42:30.234783 17386 ts_tablet_manager.cc:1428] T 25033b432cb4452096869acc626bc359 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: Time spent starting tablet: real 0.011s user 0.003s sys 0.000s
I20260810 18:42:30.237460 17383 tablet_bootstrap.cc:492] T 25033b432cb4452096869acc626bc359 P baa8dad45c5f493d92d140dea29ec5f9: No bootstrap required, opened a new log
I20260810 18:42:30.237671 17383 ts_tablet_manager.cc:1397] T 25033b432cb4452096869acc626bc359 P baa8dad45c5f493d92d140dea29ec5f9: Time spent bootstrapping tablet: real 0.033s user 0.002s sys 0.008s
I20260810 18:42:30.238394 17383 raft_consensus.cc:348] T 25033b432cb4452096869acc626bc359 P baa8dad45c5f493d92d140dea29ec5f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } }
I20260810 18:42:30.238700 17383 raft_consensus.cc:374] T 25033b432cb4452096869acc626bc359 P baa8dad45c5f493d92d140dea29ec5f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:42:30.238808 17383 raft_consensus.cc:729] T 25033b432cb4452096869acc626bc359 P baa8dad45c5f493d92d140dea29ec5f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baa8dad45c5f493d92d140dea29ec5f9, State: Initialized, Role: FOLLOWER
I20260810 18:42:30.239084 17383 consensus_queue.cc:260] T 25033b432cb4452096869acc626bc359 P baa8dad45c5f493d92d140dea29ec5f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } }
I20260810 18:42:30.239665 17383 ts_tablet_manager.cc:1428] T 25033b432cb4452096869acc626bc359 P baa8dad45c5f493d92d140dea29ec5f9: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 18:42:30.303951 17391 raft_consensus.cc:482] T 25033b432cb4452096869acc626bc359 P baa8dad45c5f493d92d140dea29ec5f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:42:30.304366 17391 raft_consensus.cc:504] T 25033b432cb4452096869acc626bc359 P baa8dad45c5f493d92d140dea29ec5f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } }
I20260810 18:42:30.305454 17391 leader_election.cc:288] T 25033b432cb4452096869acc626bc359 P baa8dad45c5f493d92d140dea29ec5f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e5373d9feb843cf8ef92ae3e7dbb5bc (127.4.228.194:43603), 0d3eb3675e6543c98972fe7cf6078944 (127.4.228.195:36567)
I20260810 18:42:30.306625 17054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25033b432cb4452096869acc626bc359" candidate_uuid: "baa8dad45c5f493d92d140dea29ec5f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" is_pre_election: true
I20260810 18:42:30.306968 17054 raft_consensus.cc:2454] T 25033b432cb4452096869acc626bc359 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate baa8dad45c5f493d92d140dea29ec5f9 in term 0.
I20260810 18:42:30.306913 17255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25033b432cb4452096869acc626bc359" candidate_uuid: "baa8dad45c5f493d92d140dea29ec5f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d3eb3675e6543c98972fe7cf6078944" is_pre_election: true
I20260810 18:42:30.307246 17255 raft_consensus.cc:2454] T 25033b432cb4452096869acc626bc359 P 0d3eb3675e6543c98972fe7cf6078944 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate baa8dad45c5f493d92d140dea29ec5f9 in term 0.
I20260810 18:42:30.307873 16729 leader_election.cc:302] T 25033b432cb4452096869acc626bc359 P baa8dad45c5f493d92d140dea29ec5f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d3eb3675e6543c98972fe7cf6078944, baa8dad45c5f493d92d140dea29ec5f9; no voters:
I20260810 18:42:30.308367 17391 raft_consensus.cc:2781] T 25033b432cb4452096869acc626bc359 P baa8dad45c5f493d92d140dea29ec5f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:42:30.308701 17391 raft_consensus.cc:482] T 25033b432cb4452096869acc626bc359 P baa8dad45c5f493d92d140dea29ec5f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:42:30.308854 17391 raft_consensus.cc:3037] T 25033b432cb4452096869acc626bc359 P baa8dad45c5f493d92d140dea29ec5f9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:42:30.311241 17391 raft_consensus.cc:504] T 25033b432cb4452096869acc626bc359 P baa8dad45c5f493d92d140dea29ec5f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } }
I20260810 18:42:30.312541 17255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25033b432cb4452096869acc626bc359" candidate_uuid: "baa8dad45c5f493d92d140dea29ec5f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d3eb3675e6543c98972fe7cf6078944"
I20260810 18:42:30.312569 17054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25033b432cb4452096869acc626bc359" candidate_uuid: "baa8dad45c5f493d92d140dea29ec5f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc"
I20260810 18:42:30.312798 17054 raft_consensus.cc:3037] T 25033b432cb4452096869acc626bc359 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:42:30.312836 17255 raft_consensus.cc:3037] T 25033b432cb4452096869acc626bc359 P 0d3eb3675e6543c98972fe7cf6078944 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:42:30.316184 17255 raft_consensus.cc:2454] T 25033b432cb4452096869acc626bc359 P 0d3eb3675e6543c98972fe7cf6078944 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate baa8dad45c5f493d92d140dea29ec5f9 in term 1.
I20260810 18:42:30.316198 17054 raft_consensus.cc:2454] T 25033b432cb4452096869acc626bc359 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate baa8dad45c5f493d92d140dea29ec5f9 in term 1.
I20260810 18:42:30.316888 16728 leader_election.cc:302] T 25033b432cb4452096869acc626bc359 P baa8dad45c5f493d92d140dea29ec5f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e5373d9feb843cf8ef92ae3e7dbb5bc, baa8dad45c5f493d92d140dea29ec5f9; no voters:
I20260810 18:42:30.317226 17391 leader_election.cc:288] T 25033b432cb4452096869acc626bc359 P baa8dad45c5f493d92d140dea29ec5f9 [CANDIDATE]: Term 1 election: Requested vote from peers 6e5373d9feb843cf8ef92ae3e7dbb5bc (127.4.228.194:43603), 0d3eb3675e6543c98972fe7cf6078944 (127.4.228.195:36567)
I20260810 18:42:30.317610 17391 raft_consensus.cc:2781] T 25033b432cb4452096869acc626bc359 P baa8dad45c5f493d92d140dea29ec5f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:42:30.317788 17391 raft_consensus.cc:686] T 25033b432cb4452096869acc626bc359 P baa8dad45c5f493d92d140dea29ec5f9 [term 1 LEADER]: Becoming Leader. State: Replica: baa8dad45c5f493d92d140dea29ec5f9, State: Running, Role: LEADER
I20260810 18:42:30.318377 17391 consensus_queue.cc:237] T 25033b432cb4452096869acc626bc359 P baa8dad45c5f493d92d140dea29ec5f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } }
I20260810 18:42:30.325223 16593 catalog_manager.cc:5495] T 25033b432cb4452096869acc626bc359 P baa8dad45c5f493d92d140dea29ec5f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to baa8dad45c5f493d92d140dea29ec5f9 (127.4.228.193). New cstate: current_term: 1 leader_uuid: "baa8dad45c5f493d92d140dea29ec5f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } health_report { overall_health: UNKNOWN } } }
I20260810 18:42:30.344944 17492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:30.335190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59452 (local address 127.4.228.193:35775)
0810 18:42:30.339036 (+ 3846us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:30.339041 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:30.339060 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:42:30.339163 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:30.339167 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:30.339256 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:30.339468 (+ 212us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:42:30.339894 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:30.340779 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:30.342349 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:30.342883 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:30.343031 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:30.343529 (+ 498us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:42:30.344165 (+ 636us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:42:30.344338 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:30.344436 (+ 98us) server_negotiation.cc:299] Negotiation successful
0810 18:42:30.344528 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3715,"thread_start_us":3544,"threads_started":1}
I20260810 18:42:30.349026 17255 raft_consensus.cc:1261] T 25033b432cb4452096869acc626bc359 P 0d3eb3675e6543c98972fe7cf6078944 [term 1 FOLLOWER]: Refusing update from remote peer baa8dad45c5f493d92d140dea29ec5f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:42:30.349086 17054 raft_consensus.cc:1261] T 25033b432cb4452096869acc626bc359 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 1 FOLLOWER]: Refusing update from remote peer baa8dad45c5f493d92d140dea29ec5f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:42:30.349866 17489 consensus_queue.cc:1035] T 25033b432cb4452096869acc626bc359 P baa8dad45c5f493d92d140dea29ec5f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:42:30.350584 17391 consensus_queue.cc:1035] T 25033b432cb4452096869acc626bc359 P baa8dad45c5f493d92d140dea29ec5f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:42:30.383536 16593 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:42:30.383878 16593 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:42:30.392357 16593 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8cfbe048354045c5b31fc749e0cfe06e: Sending DeleteTablet for 3 replicas of tablet 25033b432cb4452096869acc626bc359
I20260810 18:42:30.395179 17223 tablet_service.cc:1514] Processing DeleteTablet for tablet 25033b432cb4452096869acc626bc359 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:42:30 UTC) from {username='oryx', principal='oryx/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:60806
I20260810 18:42:30.395608 17415 tablet_replica.cc:330] T 25033b432cb4452096869acc626bc359 P 0d3eb3675e6543c98972fe7cf6078944: stopping tablet replica
I20260810 18:42:30.395830 17415 raft_consensus.cc:2229] T 25033b432cb4452096869acc626bc359 P 0d3eb3675e6543c98972fe7cf6078944 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:42:30.396087 17415 raft_consensus.cc:2258] T 25033b432cb4452096869acc626bc359 P 0d3eb3675e6543c98972fe7cf6078944 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:42:30.397662 17010 tablet_service.cc:1514] Processing DeleteTablet for tablet 25033b432cb4452096869acc626bc359 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:42:30 UTC) from {username='oryx', principal='oryx/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:42526
I20260810 18:42:30.398073 17418 tablet_replica.cc:330] T 25033b432cb4452096869acc626bc359 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: stopping tablet replica
I20260810 18:42:30.398294 17418 raft_consensus.cc:2229] T 25033b432cb4452096869acc626bc359 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:42:30.398594 17418 raft_consensus.cc:2258] T 25033b432cb4452096869acc626bc359 P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:42:30.398650 16806 tablet_service.cc:1514] Processing DeleteTablet for tablet 25033b432cb4452096869acc626bc359 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:42:30 UTC) from {username='oryx', principal='oryx/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:59436
I20260810 18:42:30.399087 16805 tablet_service.cc:1514] Processing DeleteTablet for tablet 25033b432cb4452096869acc626bc359 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:42:30 UTC) from {username='oryx', principal='oryx/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:59436
I20260810 18:42:30.399422 17416 tablet_replica.cc:330] T 25033b432cb4452096869acc626bc359 P baa8dad45c5f493d92d140dea29ec5f9: stopping tablet replica
I20260810 18:42:30.399636 17416 raft_consensus.cc:2229] T 25033b432cb4452096869acc626bc359 P baa8dad45c5f493d92d140dea29ec5f9 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:42:30.400074 17418 ts_tablet_manager.cc:1905] T 25033b432cb4452096869acc626bc359 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:42:30.405179 17415 ts_tablet_manager.cc:1905] T 25033b432cb4452096869acc626bc359 P 0d3eb3675e6543c98972fe7cf6078944: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:42:30.407758 17418 ts_tablet_manager.cc:1918] T 25033b432cb4452096869acc626bc359 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:42:30.407935 17418 log.cc:1196] T 25033b432cb4452096869acc626bc359 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-1/wal/wals/25033b432cb4452096869acc626bc359
I20260810 18:42:30.408401 17418 ts_tablet_manager.cc:1939] T 25033b432cb4452096869acc626bc359 P 6e5373d9feb843cf8ef92ae3e7dbb5bc: Deleting consensus metadata
I20260810 18:42:30.408366 16596 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60492:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:42:30.409365 16596 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:42:30.409806 16574 catalog_manager.cc:4841] TS 6e5373d9feb843cf8ef92ae3e7dbb5bc (127.4.228.194:43603): tablet 25033b432cb4452096869acc626bc359 (table test-table [id=7495e0fb1e3a4d59a99e87e19ff1860a]) successfully deleted
I20260810 18:42:30.418658 17223 tablet_service.cc:1467] Processing CreateTablet for tablet 585b401212a742189c9ab4b9122dc30a (DEFAULT_TABLE table=test-table [id=7703ae3816cb47d78785bc57f2a9a0fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:42:30.419157 17223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 585b401212a742189c9ab4b9122dc30a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:42:30.424729 17415 ts_tablet_manager.cc:1918] T 25033b432cb4452096869acc626bc359 P 0d3eb3675e6543c98972fe7cf6078944: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:42:30.424918 17415 log.cc:1196] T 25033b432cb4452096869acc626bc359 P 0d3eb3675e6543c98972fe7cf6078944: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-2/wal/wals/25033b432cb4452096869acc626bc359
I20260810 18:42:30.425431 17415 ts_tablet_manager.cc:1939] T 25033b432cb4452096869acc626bc359 P 0d3eb3675e6543c98972fe7cf6078944: Deleting consensus metadata
I20260810 18:42:30.425544 17010 tablet_service.cc:1467] Processing CreateTablet for tablet 585b401212a742189c9ab4b9122dc30a (DEFAULT_TABLE table=test-table [id=7703ae3816cb47d78785bc57f2a9a0fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:42:30.426057 17010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 585b401212a742189c9ab4b9122dc30a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:42:30.427070 17381 tablet_bootstrap.cc:492] T 585b401212a742189c9ab4b9122dc30a P 0d3eb3675e6543c98972fe7cf6078944: Bootstrap starting.
I20260810 18:42:30.428556 16805 tablet_service.cc:1467] Processing CreateTablet for tablet 585b401212a742189c9ab4b9122dc30a (DEFAULT_TABLE table=test-table [id=7703ae3816cb47d78785bc57f2a9a0fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:42:30.428769 17381 tablet_bootstrap.cc:654] T 585b401212a742189c9ab4b9122dc30a P 0d3eb3675e6543c98972fe7cf6078944: Neither blocks nor log segments found. Creating new log.
I20260810 18:42:30.429272 16805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 585b401212a742189c9ab4b9122dc30a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:42:30.430229 16575 catalog_manager.cc:4841] TS 0d3eb3675e6543c98972fe7cf6078944 (127.4.228.195:36567): tablet 25033b432cb4452096869acc626bc359 (table test-table [id=7495e0fb1e3a4d59a99e87e19ff1860a]) successfully deleted
I20260810 18:42:30.431103 17381 tablet_bootstrap.cc:492] T 585b401212a742189c9ab4b9122dc30a P 0d3eb3675e6543c98972fe7cf6078944: No bootstrap required, opened a new log
I20260810 18:42:30.431299 17381 ts_tablet_manager.cc:1397] T 585b401212a742189c9ab4b9122dc30a P 0d3eb3675e6543c98972fe7cf6078944: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20260810 18:42:30.431951 17381 raft_consensus.cc:348] T 585b401212a742189c9ab4b9122dc30a P 0d3eb3675e6543c98972fe7cf6078944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } }
I20260810 18:42:30.432201 17381 raft_consensus.cc:374] T 585b401212a742189c9ab4b9122dc30a P 0d3eb3675e6543c98972fe7cf6078944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:42:30.432308 17381 raft_consensus.cc:729] T 585b401212a742189c9ab4b9122dc30a P 0d3eb3675e6543c98972fe7cf6078944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d3eb3675e6543c98972fe7cf6078944, State: Initialized, Role: FOLLOWER
I20260810 18:42:30.432616 17381 consensus_queue.cc:260] T 585b401212a742189c9ab4b9122dc30a P 0d3eb3675e6543c98972fe7cf6078944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } }
I20260810 18:42:30.433126 17381 ts_tablet_manager.cc:1428] T 585b401212a742189c9ab4b9122dc30a P 0d3eb3675e6543c98972fe7cf6078944: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260810 18:42:30.438263 17383 tablet_bootstrap.cc:492] T 585b401212a742189c9ab4b9122dc30a P baa8dad45c5f493d92d140dea29ec5f9: Bootstrap starting.
I20260810 18:42:30.440268 17383 tablet_bootstrap.cc:654] T 585b401212a742189c9ab4b9122dc30a P baa8dad45c5f493d92d140dea29ec5f9: Neither blocks nor log segments found. Creating new log.
I20260810 18:42:30.447631 17416 raft_consensus.cc:2258] T 25033b432cb4452096869acc626bc359 P baa8dad45c5f493d92d140dea29ec5f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:42:30.448997 17416 ts_tablet_manager.cc:1905] T 25033b432cb4452096869acc626bc359 P baa8dad45c5f493d92d140dea29ec5f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:42:30.456542 17416 ts_tablet_manager.cc:1918] T 25033b432cb4452096869acc626bc359 P baa8dad45c5f493d92d140dea29ec5f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:42:30.456717 17416 log.cc:1196] T 25033b432cb4452096869acc626bc359 P baa8dad45c5f493d92d140dea29ec5f9: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-0/wal/wals/25033b432cb4452096869acc626bc359
I20260810 18:42:30.457226 17416 ts_tablet_manager.cc:1939] T 25033b432cb4452096869acc626bc359 P baa8dad45c5f493d92d140dea29ec5f9: Deleting consensus metadata
I20260810 18:42:30.458456 17383 tablet_bootstrap.cc:492] T 585b401212a742189c9ab4b9122dc30a P baa8dad45c5f493d92d140dea29ec5f9: No bootstrap required, opened a new log
I20260810 18:42:30.458745 17383 ts_tablet_manager.cc:1397] T 585b401212a742189c9ab4b9122dc30a P baa8dad45c5f493d92d140dea29ec5f9: Time spent bootstrapping tablet: real 0.021s user 0.004s sys 0.006s
I20260810 18:42:30.458709 16574 catalog_manager.cc:4841] TS baa8dad45c5f493d92d140dea29ec5f9 (127.4.228.193:35775): tablet 25033b432cb4452096869acc626bc359 (table test-table [id=7495e0fb1e3a4d59a99e87e19ff1860a]) successfully deleted
W20260810 18:42:30.459844 16574 catalog_manager.cc:4808] TS baa8dad45c5f493d92d140dea29ec5f9 (127.4.228.193:35775): delete failed for tablet 25033b432cb4452096869acc626bc359 because the tablet was not found. No further retry: Not found: Tablet not found: 25033b432cb4452096869acc626bc359
I20260810 18:42:30.459775 17383 raft_consensus.cc:348] T 585b401212a742189c9ab4b9122dc30a P baa8dad45c5f493d92d140dea29ec5f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } }
I20260810 18:42:30.460268 17383 raft_consensus.cc:374] T 585b401212a742189c9ab4b9122dc30a P baa8dad45c5f493d92d140dea29ec5f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:42:30.460435 17383 raft_consensus.cc:729] T 585b401212a742189c9ab4b9122dc30a P baa8dad45c5f493d92d140dea29ec5f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baa8dad45c5f493d92d140dea29ec5f9, State: Initialized, Role: FOLLOWER
I20260810 18:42:30.460844 17383 consensus_queue.cc:260] T 585b401212a742189c9ab4b9122dc30a P baa8dad45c5f493d92d140dea29ec5f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } }
I20260810 18:42:30.461942 17386 tablet_bootstrap.cc:492] T 585b401212a742189c9ab4b9122dc30a P 6e5373d9feb843cf8ef92ae3e7dbb5bc: Bootstrap starting.
I20260810 18:42:30.464008 17386 tablet_bootstrap.cc:654] T 585b401212a742189c9ab4b9122dc30a P 6e5373d9feb843cf8ef92ae3e7dbb5bc: Neither blocks nor log segments found. Creating new log.
I20260810 18:42:30.464465 17383 ts_tablet_manager.cc:1428] T 585b401212a742189c9ab4b9122dc30a P baa8dad45c5f493d92d140dea29ec5f9: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260810 18:42:30.473748 17458 raft_consensus.cc:482] T 585b401212a742189c9ab4b9122dc30a P 0d3eb3675e6543c98972fe7cf6078944 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:42:30.474016 17458 raft_consensus.cc:504] T 585b401212a742189c9ab4b9122dc30a P 0d3eb3675e6543c98972fe7cf6078944 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } }
I20260810 18:42:30.474921 17458 leader_election.cc:288] T 585b401212a742189c9ab4b9122dc30a P 0d3eb3675e6543c98972fe7cf6078944 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e5373d9feb843cf8ef92ae3e7dbb5bc (127.4.228.194:43603), baa8dad45c5f493d92d140dea29ec5f9 (127.4.228.193:35775)
I20260810 18:42:30.476223 16840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "585b401212a742189c9ab4b9122dc30a" candidate_uuid: "0d3eb3675e6543c98972fe7cf6078944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa8dad45c5f493d92d140dea29ec5f9" is_pre_election: true
I20260810 18:42:30.476226 17053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "585b401212a742189c9ab4b9122dc30a" candidate_uuid: "0d3eb3675e6543c98972fe7cf6078944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" is_pre_election: true
I20260810 18:42:30.476518 16840 raft_consensus.cc:2454] T 585b401212a742189c9ab4b9122dc30a P baa8dad45c5f493d92d140dea29ec5f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d3eb3675e6543c98972fe7cf6078944 in term 0.
I20260810 18:42:30.476845 17386 tablet_bootstrap.cc:492] T 585b401212a742189c9ab4b9122dc30a P 6e5373d9feb843cf8ef92ae3e7dbb5bc: No bootstrap required, opened a new log
I20260810 18:42:30.477064 17386 ts_tablet_manager.cc:1397] T 585b401212a742189c9ab4b9122dc30a P 6e5373d9feb843cf8ef92ae3e7dbb5bc: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.002s
I20260810 18:42:30.477502 17152 leader_election.cc:302] T 585b401212a742189c9ab4b9122dc30a P 0d3eb3675e6543c98972fe7cf6078944 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d3eb3675e6543c98972fe7cf6078944, baa8dad45c5f493d92d140dea29ec5f9; no voters:
I20260810 18:42:30.478147 17386 raft_consensus.cc:348] T 585b401212a742189c9ab4b9122dc30a P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } }
W20260810 18:42:30.478226 17152 leader_election.cc:341] T 585b401212a742189c9ab4b9122dc30a P 0d3eb3675e6543c98972fe7cf6078944 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6e5373d9feb843cf8ef92ae3e7dbb5bc (127.4.228.194:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260810 18:42:30.478418 17386 raft_consensus.cc:374] T 585b401212a742189c9ab4b9122dc30a P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:42:30.478542 17386 raft_consensus.cc:729] T 585b401212a742189c9ab4b9122dc30a P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e5373d9feb843cf8ef92ae3e7dbb5bc, State: Initialized, Role: FOLLOWER
I20260810 18:42:30.478823 17386 consensus_queue.cc:260] T 585b401212a742189c9ab4b9122dc30a P 6e5373d9feb843cf8ef92ae3e7dbb5bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } }
I20260810 18:42:30.478982 17458 raft_consensus.cc:2781] T 585b401212a742189c9ab4b9122dc30a P 0d3eb3675e6543c98972fe7cf6078944 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:42:30.479125 17458 raft_consensus.cc:482] T 585b401212a742189c9ab4b9122dc30a P 0d3eb3675e6543c98972fe7cf6078944 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:42:30.479221 17458 raft_consensus.cc:3037] T 585b401212a742189c9ab4b9122dc30a P 0d3eb3675e6543c98972fe7cf6078944 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:42:30.479364 17386 ts_tablet_manager.cc:1428] T 585b401212a742189c9ab4b9122dc30a P 6e5373d9feb843cf8ef92ae3e7dbb5bc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 18:42:30.485805 17458 raft_consensus.cc:504] T 585b401212a742189c9ab4b9122dc30a P 0d3eb3675e6543c98972fe7cf6078944 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } }
I20260810 18:42:30.487218 17053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "585b401212a742189c9ab4b9122dc30a" candidate_uuid: "0d3eb3675e6543c98972fe7cf6078944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc"
I20260810 18:42:30.487512 17053 raft_consensus.cc:3037] T 585b401212a742189c9ab4b9122dc30a P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:42:30.491060 17053 raft_consensus.cc:2454] T 585b401212a742189c9ab4b9122dc30a P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d3eb3675e6543c98972fe7cf6078944 in term 1.
I20260810 18:42:30.491971 17152 leader_election.cc:302] T 585b401212a742189c9ab4b9122dc30a P 0d3eb3675e6543c98972fe7cf6078944 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d3eb3675e6543c98972fe7cf6078944, 6e5373d9feb843cf8ef92ae3e7dbb5bc; no voters:
I20260810 18:42:30.494354 17458 leader_election.cc:288] T 585b401212a742189c9ab4b9122dc30a P 0d3eb3675e6543c98972fe7cf6078944 [CANDIDATE]: Term 1 election: Requested vote from peers 6e5373d9feb843cf8ef92ae3e7dbb5bc (127.4.228.194:43603), baa8dad45c5f493d92d140dea29ec5f9 (127.4.228.193:35775)
I20260810 18:42:30.495383 17458 raft_consensus.cc:2781] T 585b401212a742189c9ab4b9122dc30a P 0d3eb3675e6543c98972fe7cf6078944 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:42:30.495843 17458 raft_consensus.cc:686] T 585b401212a742189c9ab4b9122dc30a P 0d3eb3675e6543c98972fe7cf6078944 [term 1 LEADER]: Becoming Leader. State: Replica: 0d3eb3675e6543c98972fe7cf6078944, State: Running, Role: LEADER
I20260810 18:42:30.496688 17458 consensus_queue.cc:237] T 585b401212a742189c9ab4b9122dc30a P 0d3eb3675e6543c98972fe7cf6078944 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } } peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } }
I20260810 18:42:30.516613 16840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "585b401212a742189c9ab4b9122dc30a" candidate_uuid: "0d3eb3675e6543c98972fe7cf6078944" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa8dad45c5f493d92d140dea29ec5f9"
I20260810 18:42:30.516940 16840 raft_consensus.cc:3037] T 585b401212a742189c9ab4b9122dc30a P baa8dad45c5f493d92d140dea29ec5f9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:42:30.520300 16840 raft_consensus.cc:2454] T 585b401212a742189c9ab4b9122dc30a P baa8dad45c5f493d92d140dea29ec5f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d3eb3675e6543c98972fe7cf6078944 in term 1.
I20260810 18:42:30.522153 16596 catalog_manager.cc:5495] T 585b401212a742189c9ab4b9122dc30a P 0d3eb3675e6543c98972fe7cf6078944 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d3eb3675e6543c98972fe7cf6078944 (127.4.228.195). New cstate: current_term: 1 leader_uuid: "0d3eb3675e6543c98972fe7cf6078944" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d3eb3675e6543c98972fe7cf6078944" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36567 } health_report { overall_health: HEALTHY } } }
I20260810 18:42:30.591080 17506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:30.554655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60826 (local address 127.4.228.195:36567)
0810 18:42:30.574536 (+ 19881us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:30.574545 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:30.574574 (+ 29us) server_negotiation.cc:407] Connection header received
0810 18:42:30.574672 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:30.574679 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:30.575048 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:30.580769 (+ 5721us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:42:30.581153 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:30.582328 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:30.584728 (+ 2400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:30.585488 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:30.589540 (+ 4052us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:30.589695 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:42:30.589868 (+ 173us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:42:30.590590 (+ 722us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:30.590644 (+ 54us) server_negotiation.cc:299] Negotiation successful
0810 18:42:30.590777 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19649,"thread_start_us":20695,"threads_started":1}
I20260810 18:42:30.614714 16840 raft_consensus.cc:1261] T 585b401212a742189c9ab4b9122dc30a P baa8dad45c5f493d92d140dea29ec5f9 [term 1 FOLLOWER]: Refusing update from remote peer 0d3eb3675e6543c98972fe7cf6078944: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:42:30.614847 17053 raft_consensus.cc:1261] T 585b401212a742189c9ab4b9122dc30a P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 1 FOLLOWER]: Refusing update from remote peer 0d3eb3675e6543c98972fe7cf6078944: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:42:30.615734 17458 consensus_queue.cc:1035] T 585b401212a742189c9ab4b9122dc30a P 0d3eb3675e6543c98972fe7cf6078944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "baa8dad45c5f493d92d140dea29ec5f9" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 35775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:42:30.616052 17502 consensus_queue.cc:1035] T 585b401212a742189c9ab4b9122dc30a P 0d3eb3675e6543c98972fe7cf6078944 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e5373d9feb843cf8ef92ae3e7dbb5bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:42:30.762117 16596 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:42:30.762442 16596 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:42:30.766482 16596 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8cfbe048354045c5b31fc749e0cfe06e: Sending DeleteTablet for 3 replicas of tablet 585b401212a742189c9ab4b9122dc30a
I20260810 18:42:30.767716 17010 tablet_service.cc:1514] Processing DeleteTablet for tablet 585b401212a742189c9ab4b9122dc30a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:42:30 UTC) from {username='oryx', principal='oryx/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:42526
I20260810 18:42:30.767745 16805 tablet_service.cc:1514] Processing DeleteTablet for tablet 585b401212a742189c9ab4b9122dc30a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:42:30 UTC) from {username='oryx', principal='oryx/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:59436
I20260810 18:42:30.768067 17416 tablet_replica.cc:330] T 585b401212a742189c9ab4b9122dc30a P baa8dad45c5f493d92d140dea29ec5f9: stopping tablet replica
I20260810 18:42:30.768085 17418 tablet_replica.cc:330] T 585b401212a742189c9ab4b9122dc30a P 6e5373d9feb843cf8ef92ae3e7dbb5bc: stopping tablet replica
I20260810 18:42:30.768313 17418 raft_consensus.cc:2229] T 585b401212a742189c9ab4b9122dc30a P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:42:30.768311 17416 raft_consensus.cc:2229] T 585b401212a742189c9ab4b9122dc30a P baa8dad45c5f493d92d140dea29ec5f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:42:30.768534 17416 raft_consensus.cc:2258] T 585b401212a742189c9ab4b9122dc30a P baa8dad45c5f493d92d140dea29ec5f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:42:30.768545 17418 raft_consensus.cc:2258] T 585b401212a742189c9ab4b9122dc30a P 6e5373d9feb843cf8ef92ae3e7dbb5bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:42:30.769482 17223 tablet_service.cc:1514] Processing DeleteTablet for tablet 585b401212a742189c9ab4b9122dc30a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:42:30 UTC) from {username='oryx', principal='oryx/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:60806
I20260810 18:42:30.769920 17415 tablet_replica.cc:330] T 585b401212a742189c9ab4b9122dc30a P 0d3eb3675e6543c98972fe7cf6078944: stopping tablet replica
I20260810 18:42:30.770186 17415 raft_consensus.cc:2229] T 585b401212a742189c9ab4b9122dc30a P 0d3eb3675e6543c98972fe7cf6078944 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:42:30.770576 17418 ts_tablet_manager.cc:1905] T 585b401212a742189c9ab4b9122dc30a P 6e5373d9feb843cf8ef92ae3e7dbb5bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:42:30.770601 17415 raft_consensus.cc:2258] T 585b401212a742189c9ab4b9122dc30a P 0d3eb3675e6543c98972fe7cf6078944 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:42:30.775559 17415 ts_tablet_manager.cc:1905] T 585b401212a742189c9ab4b9122dc30a P 0d3eb3675e6543c98972fe7cf6078944: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:42:30.775913 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 16703
I20260810 18:42:30.780851 17416 ts_tablet_manager.cc:1905] T 585b401212a742189c9ab4b9122dc30a P baa8dad45c5f493d92d140dea29ec5f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:42:30.787421 17415 ts_tablet_manager.cc:1918] T 585b401212a742189c9ab4b9122dc30a P 0d3eb3675e6543c98972fe7cf6078944: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:42:30.787627 17415 log.cc:1196] T 585b401212a742189c9ab4b9122dc30a P 0d3eb3675e6543c98972fe7cf6078944: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-2/wal/wals/585b401212a742189c9ab4b9122dc30a
I20260810 18:42:30.788102 17415 ts_tablet_manager.cc:1939] T 585b401212a742189c9ab4b9122dc30a P 0d3eb3675e6543c98972fe7cf6078944: Deleting consensus metadata
I20260810 18:42:30.789462 16575 catalog_manager.cc:4841] TS 0d3eb3675e6543c98972fe7cf6078944 (127.4.228.195:36567): tablet 585b401212a742189c9ab4b9122dc30a (table test-table [id=7703ae3816cb47d78785bc57f2a9a0fd]) successfully deleted
I20260810 18:42:30.799942 17418 ts_tablet_manager.cc:1918] T 585b401212a742189c9ab4b9122dc30a P 6e5373d9feb843cf8ef92ae3e7dbb5bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:42:30.803593 17416 ts_tablet_manager.cc:1918] T 585b401212a742189c9ab4b9122dc30a P baa8dad45c5f493d92d140dea29ec5f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:42:30.810685 17418 log.cc:1196] T 585b401212a742189c9ab4b9122dc30a P 6e5373d9feb843cf8ef92ae3e7dbb5bc: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-1/wal/wals/585b401212a742189c9ab4b9122dc30a
Aug 10 18:42:30 dist-test-slave-7kzw krb5kdc[16534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.228.193@KRBTEST.COM
I20260810 18:42:30.811429 17418 ts_tablet_manager.cc:1939] T 585b401212a742189c9ab4b9122dc30a P 6e5373d9feb843cf8ef92ae3e7dbb5bc: Deleting consensus metadata
I20260810 18:42:30.812825 16574 catalog_manager.cc:4841] TS 6e5373d9feb843cf8ef92ae3e7dbb5bc (127.4.228.194:43603): tablet 585b401212a742189c9ab4b9122dc30a (table test-table [id=7703ae3816cb47d78785bc57f2a9a0fd]) successfully deleted
I20260810 18:42:30.813319 17416 log.cc:1196] T 585b401212a742189c9ab4b9122dc30a P baa8dad45c5f493d92d140dea29ec5f9: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387267874959-5011-0/minicluster-data/ts-0/wal/wals/585b401212a742189c9ab4b9122dc30a
I20260810 18:42:30.814014 17416 ts_tablet_manager.cc:1939] T 585b401212a742189c9ab4b9122dc30a P baa8dad45c5f493d92d140dea29ec5f9: Deleting consensus metadata
I20260810 18:42:30.815418 16574 catalog_manager.cc:4841] TS baa8dad45c5f493d92d140dea29ec5f9 (127.4.228.193:35775): tablet 585b401212a742189c9ab4b9122dc30a (table test-table [id=7703ae3816cb47d78785bc57f2a9a0fd]) successfully deleted
I20260810 18:42:30.822536 17492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:30.779406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59454 (local address 127.4.228.193:35775)
0810 18:42:30.779601 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:30.779607 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:30.799198 (+ 19591us) server_negotiation.cc:407] Connection header received
0810 18:42:30.799761 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:30.799769 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:30.799886 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:30.799976 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:30.801924 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:30.803253 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:30.805595 (+ 2342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:30.806299 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:30.806415 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:30.812758 (+ 6343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:30.812780 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:30.812804 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:30.812858 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:30.817575 (+ 4717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:30.819201 (+ 1626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:30.819214 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:30.819219 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:30.819432 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:30.820009 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:30.820015 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:30.820019 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:30.820552 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:30.821014 (+ 462us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:30.821787 (+ 773us) server_negotiation.cc:299] Negotiation successful
0810 18:42:30.822080 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 18:42:30.824501 16901 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:42:30.955343 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 16703
I20260810 18:42:30.989528 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 16911
Aug 10 18:42:30 dist-test-slave-7kzw krb5kdc[16534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.228.194@KRBTEST.COM
I20260810 18:42:31.009034 17520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:30.990508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42560 (local address 127.4.228.194:43603)
0810 18:42:30.991104 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:30.991109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:30.991129 (+ 20us) server_negotiation.cc:407] Connection header received
0810 18:42:30.991193 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:30.991197 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:30.991303 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:30.991445 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:30.993081 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:30.993899 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:30.995632 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:30.996078 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:30.996200 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:31.001479 (+ 5279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:31.001499 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:31.001520 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:31.001568 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:31.005551 (+ 3983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:31.006451 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:31.006459 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:31.006514 (+ 55us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:31.006713 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:31.007147 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:31.007152 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:31.007157 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:31.007596 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:31.007962 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:31.008433 (+ 471us) server_negotiation.cc:299] Negotiation successful
0810 18:42:31.008672 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":290,"threads_started":1}
I20260810 18:42:31.009971 17119 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:42:31.167032 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 16911
W20260810 18:42:31.202147 17152 connection.cc:535] server connection from 127.4.228.194:49861 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 18:42:31.202208 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 17131
Aug 10 18:42:31 dist-test-slave-7kzw krb5kdc[16534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.228.195@KRBTEST.COM
I20260810 18:42:31.220757 17522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:31.203504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60842 (local address 127.4.228.195:36567)
0810 18:42:31.204126 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:31.204130 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:31.204153 (+ 23us) server_negotiation.cc:407] Connection header received
0810 18:42:31.204210 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:31.204214 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:31.204276 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:31.204404 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:31.205831 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:31.206647 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:31.208091 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:31.208531 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:31.208677 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:31.213671 (+ 4994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:31.213691 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:31.213710 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:31.213760 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:31.217125 (+ 3365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:31.218057 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:31.218067 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:31.218069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:31.218242 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:31.218755 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:31.218760 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:31.218766 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:31.219213 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:31.219581 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:31.220080 (+ 499us) server_negotiation.cc:299] Negotiation successful
0810 18:42:31.220365 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":359,"threads_started":1}
I20260810 18:42:31.221724 17324 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:42:31.351665 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 17131
I20260810 18:42:31.388355 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 16552
I20260810 18:42:31.389825 16658 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:42:31.515911 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 16552
2026-08-10T18:42:31Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (7716 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:42:31 dist-test-slave-7kzw krb5kdc[17534](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:42:31 dist-test-slave-7kzw krb5kdc[17534](info): set up 2 sockets
Aug 10 18:42:31 dist-test-slave-7kzw krb5kdc[17534](info): commencing operation
krb5kdc: starting...
W20260810 18:42:32.920373 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.341s user 0.002s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:42:33 dist-test-slave-7kzw krb5kdc[17534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:42:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:42:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.228.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
I20260810 18:42:33.244100 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:33457
--webserver_interface=127.4.228.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:45107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:33457
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:42:33.623693 17552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:42:33.624068 17552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:42:33.624186 17552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:42:33.632459 17552 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:42:33.632603 17552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:42:33.632820 17552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:42:33.632956 17552 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:42:33.633021 17552 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:42:33.644498 17552 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:45107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:33457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:33457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:42:33.645977 17552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:42:33.647897 17552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:42:33.660959 17561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:33.661149 17563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:33.661371 17560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:42:33.664419 17552 server_base.cc:1034] running on GCE node
I20260810 18:42:33.665427 17552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:42:33.667507 17552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:42:33.668753 17552 hybrid_clock.cc:648] HybridClock initialized: now 1786387353668693 us; error 49 us; skew 500 ppm
I20260810 18:42:33.669101 17552 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-10T18:42:33Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2209 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:42:33 dist-test-slave-7kzw krb5kdc[17573](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:42:33 dist-test-slave-7kzw krb5kdc[17573](info): set up 2 sockets
Aug 10 18:42:33 dist-test-slave-7kzw krb5kdc[17573](info): commencing operation
krb5kdc: starting...
W20260810 18:42:35.134096 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.345s user 0.003s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:42:35 dist-test-slave-7kzw krb5kdc[17573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:42:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:42:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.228.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
I20260810 18:42:35.475814 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:35243
--webserver_interface=127.4.228.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:46853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:35243
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:42:35.853895 17590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:42:35.854290 17590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:42:35.854453 17590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:42:35.862293 17590 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:42:35.862439 17590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:42:35.862512 17590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:42:35.862608 17590 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:42:35.862682 17590 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:42:35.873415 17590 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:46853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:35243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:35243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:42:35.875008 17590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:42:35.876498 17590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:42:35.890698 17596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:35.890604 17597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:35.890673 17601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:42:35.892025 17590 server_base.cc:1034] running on GCE node
I20260810 18:42:35.893079 17590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:42:35.894933 17590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:42:35.896265 17590 hybrid_clock.cc:648] HybridClock initialized: now 1786387355896178 us; error 68 us; skew 500 ppm
I20260810 18:42:35.896812 17590 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:42:35 dist-test-slave-7kzw krb5kdc[17573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387355, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:42:35.906760 17590 init.cc:377] Logged in from keytab as kudu/127.4.228.254@KRBTEST.COM (short username kudu)
I20260810 18:42:35.908715 17590 webserver.cc:460] Webserver started at http://127.4.228.254:33435/ using document root <none> and password file <none>
I20260810 18:42:35.909540 17590 fs_manager.cc:359] Metadata directory not provided
I20260810 18:42:35.909669 17590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:42:35.909970 17590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:42:35.913465 17590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "df7a4c4084074982824c194343d2d7a8"
format_stamp: "Formatted at 2026-08-10 18:42:35 on dist-test-slave-7kzw"
server_key: "64173377726de776ae16f8f276d65cd1"
server_key_iv: "34c6db46f2cd59ce20da20b7936c2afc"
server_key_version: "encryptionkey@0"
I20260810 18:42:35.914342 17590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "df7a4c4084074982824c194343d2d7a8"
format_stamp: "Formatted at 2026-08-10 18:42:35 on dist-test-slave-7kzw"
server_key: "64173377726de776ae16f8f276d65cd1"
server_key_iv: "34c6db46f2cd59ce20da20b7936c2afc"
server_key_version: "encryptionkey@0"
I20260810 18:42:35.920593 17590 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 18:42:35.925222 17616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:35.927287 17590 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260810 18:42:35.927493 17590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "df7a4c4084074982824c194343d2d7a8"
format_stamp: "Formatted at 2026-08-10 18:42:35 on dist-test-slave-7kzw"
server_key: "64173377726de776ae16f8f276d65cd1"
server_key_iv: "34c6db46f2cd59ce20da20b7936c2afc"
server_key_version: "encryptionkey@0"
I20260810 18:42:35.927703 17590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:42:35.946741 17590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:42:35.952277 17590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:42:35.952600 17590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:42:35.975313 17590 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:35243
I20260810 18:42:35.975327 17692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:35243 every 8 connection(s)
I20260810 18:42:35.977072 17590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:42:35.979599 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 17590
I20260810 18:42:35.979982 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:42:35.981168 5011 external_mini_cluster.cc:1428] Setting key 4e3d195d5847cd5c843cd2d85cfc76fb
I20260810 18:42:35.982591 17693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:42:35.997393 17693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8: Bootstrap starting.
Aug 10 18:42:35 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:42:36.003125 17693 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8: Neither blocks nor log segments found. Creating new log.
I20260810 18:42:36.004498 17693 log.cc:824] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8: Log is configured to *not* fsync() on all Append() calls
I20260810 18:42:36.008421 17693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8: No bootstrap required, opened a new log
I20260810 18:42:36.008705 17697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:35.983695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32790 (local address 127.4.228.254:35243)
0810 18:42:35.984856 (+ 1161us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:35.984881 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:35.984933 (+ 52us) server_negotiation.cc:407] Connection header received
0810 18:42:35.986751 (+ 1818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:35.986777 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:35.987215 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:35.987715 (+ 500us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:35.989721 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:35.991629 (+ 1908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:35.993177 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:35.993977 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:35.994243 (+ 266us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:35.999023 (+ 4780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:35.999032 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:35.999050 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:35.999102 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:36.003633 (+ 4531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:36.004552 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:36.004560 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:36.004568 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:36.004830 (+ 262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:36.005269 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:36.005274 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:36.005278 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:36.005957 (+ 679us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:36.006306 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:36.006887 (+ 581us) server_negotiation.cc:299] Negotiation successful
0810 18:42:36.007434 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":721,"thread_start_us":460,"threads_started":1}
I20260810 18:42:36.018381 17693 raft_consensus.cc:348] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df7a4c4084074982824c194343d2d7a8" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35243 } }
I20260810 18:42:36.018960 17693 raft_consensus.cc:374] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:42:36.019112 17693 raft_consensus.cc:729] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df7a4c4084074982824c194343d2d7a8, State: Initialized, Role: FOLLOWER
I20260810 18:42:36.019871 17693 consensus_queue.cc:260] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df7a4c4084074982824c194343d2d7a8" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35243 } }
I20260810 18:42:36.020112 17693 raft_consensus.cc:388] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:42:36.020216 17693 raft_consensus.cc:482] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:42:36.020356 17693 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:42:36.023221 17693 raft_consensus.cc:504] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df7a4c4084074982824c194343d2d7a8" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35243 } }
I20260810 18:42:36.023856 17693 leader_election.cc:302] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df7a4c4084074982824c194343d2d7a8; no voters:
I20260810 18:42:36.024775 17693 leader_election.cc:288] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:42:36.024933 17700 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:42:36.025978 17700 raft_consensus.cc:686] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8 [term 1 LEADER]: Becoming Leader. State: Replica: df7a4c4084074982824c194343d2d7a8, State: Running, Role: LEADER
I20260810 18:42:36.026820 17700 consensus_queue.cc:237] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df7a4c4084074982824c194343d2d7a8" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35243 } }
I20260810 18:42:36.028223 17693 sys_catalog.cc:564] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:42:36.030599 17703 sys_catalog.cc:455] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df7a4c4084074982824c194343d2d7a8. Latest consensus state: current_term: 1 leader_uuid: "df7a4c4084074982824c194343d2d7a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df7a4c4084074982824c194343d2d7a8" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35243 } } }
I20260810 18:42:36.032002 17703 sys_catalog.cc:458] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8 [sys.catalog]: This master's current role is: LEADER
I20260810 18:42:36.034879 17702 sys_catalog.cc:455] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df7a4c4084074982824c194343d2d7a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df7a4c4084074982824c194343d2d7a8" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35243 } } }
I20260810 18:42:36.035456 17702 sys_catalog.cc:458] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8 [sys.catalog]: This master's current role is: LEADER
I20260810 18:42:36.035996 17708 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:42:36.042186 17708 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:42:36.055588 17708 catalog_manager.cc:1347] Generated new cluster ID: c1182782702f4728b39064340fbbea3c
I20260810 18:42:36.055811 17708 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:42:36.077654 17708 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:42:36.079970 17708 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:42:36.091502 17708 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8: Generated new TSK 0
I20260810 18:42:36.092904 17708 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.228.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab.
Entry for principal kudu/127.4.228.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab.
I20260810 18:42:36.171608 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.193:0
--local_ip_for_outbound_sockets=127.4.228.193
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:35243
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:46853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:42:36.557875 17727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:42:36.558219 17727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:42:36.558320 17727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:42:36.565779 17727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:42:36.565907 17727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:42:36.566023 17727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.193
I20260810 18:42:36.576220 17727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:46853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:35243
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.193
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:42:36.577623 17727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:42:36.579066 17727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:42:36.593915 17742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:36.594118 17735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:36.594105 17736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:42:36.595623 17727 server_base.cc:1034] running on GCE node
I20260810 18:42:36.596490 17727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:42:36.597816 17727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:42:36.599126 17727 hybrid_clock.cc:648] HybridClock initialized: now 1786387356599010 us; error 102 us; skew 500 ppm
I20260810 18:42:36.599568 17727 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:42:36 dist-test-slave-7kzw krb5kdc[17573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387356, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:42:36.603669 17727 init.cc:377] Logged in from keytab as kudu/127.4.228.193@KRBTEST.COM (short username kudu)
I20260810 18:42:36.605504 17727 webserver.cc:460] Webserver started at http://127.4.228.193:33317/ using document root <none> and password file <none>
I20260810 18:42:36.606367 17727 fs_manager.cc:359] Metadata directory not provided
I20260810 18:42:36.606556 17727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:42:36.606952 17727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:42:36.610405 17727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-0/data/instance:
uuid: "674a0186f31945e7805c6b9d629d493b"
format_stamp: "Formatted at 2026-08-10 18:42:36 on dist-test-slave-7kzw"
server_key: "f9e9303f29883226380ae4542f57a96d"
server_key_iv: "93bdda39d1e2d5a8da6b071c90d56178"
server_key_version: "encryptionkey@0"
I20260810 18:42:36.611505 17727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance:
uuid: "674a0186f31945e7805c6b9d629d493b"
format_stamp: "Formatted at 2026-08-10 18:42:36 on dist-test-slave-7kzw"
server_key: "f9e9303f29883226380ae4542f57a96d"
server_key_iv: "93bdda39d1e2d5a8da6b071c90d56178"
server_key_version: "encryptionkey@0"
I20260810 18:42:36.617753 17727 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.002s
I20260810 18:42:36.621846 17756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:36.623502 17727 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 18:42:36.623700 17727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-0/wal
uuid: "674a0186f31945e7805c6b9d629d493b"
format_stamp: "Formatted at 2026-08-10 18:42:36 on dist-test-slave-7kzw"
server_key: "f9e9303f29883226380ae4542f57a96d"
server_key_iv: "93bdda39d1e2d5a8da6b071c90d56178"
server_key_version: "encryptionkey@0"
I20260810 18:42:36.623943 17727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:42:36.639544 17727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:42:36.644791 17727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:42:36.645180 17727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:42:36.646521 17727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:42:36.648303 17727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:42:36.648401 17727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:36.648559 17727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:42:36.648644 17727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:36.686050 17727 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.193:41101
I20260810 18:42:36.686094 17923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.193:41101 every 8 connection(s)
I20260810 18:42:36.687539 17727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
I20260810 18:42:36.695171 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 17727
I20260810 18:42:36.695426 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance
I20260810 18:42:36.696111 5011 external_mini_cluster.cc:1428] Setting key d3c31a1503a2180c1220ce7e057d8347
Aug 10 18:42:36 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387356, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:42:36.709720 17933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:36.690588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:34631 (local address 127.4.228.254:35243)
0810 18:42:36.691096 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:36.691101 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:36.692040 (+ 939us) server_negotiation.cc:407] Connection header received
0810 18:42:36.693983 (+ 1943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:36.693988 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:36.694086 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:36.694212 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:36.696154 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:36.697066 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:36.698833 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:36.699259 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:36.699406 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:36.703404 (+ 3998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:36.703410 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:36.703413 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:36.703450 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:36.706192 (+ 2742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:36.707095 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:36.707103 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:36.707113 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:36.707242 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:36.707740 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:36.707744 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:36.707746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:36.708034 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:36.708200 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:36.708985 (+ 785us) server_negotiation.cc:299] Negotiation successful
0810 18:42:36.709228 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":271,"threads_started":1}
I20260810 18:42:36.710922 17934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:36.691128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.193:34631)
0810 18:42:36.691871 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:36.691912 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:36.693695 (+ 1783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:36.694556 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:36.694565 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:36.695180 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:36.695957 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:36.695976 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:36.697243 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:36.697247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:36.698670 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:36.698684 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:36.699524 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:36.699532 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:36.699783 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:36.700532 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:36.700558 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:36.703137 (+ 2579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:36.706358 (+ 3221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:36.706368 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:36.706388 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:36.706939 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:36.707378 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:36.707382 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:36.707386 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:36.707605 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:36.708180 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:36.708188 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:36.708568 (+ 380us) client_negotiation.cc:769] Sending connection context
0810 18:42:36.708862 (+ 294us) client_negotiation.cc:240] Negotiation successful
0810 18:42:36.709461 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":281,"threads_started":1}
I20260810 18:42:36.712807 17925 heartbeater.cc:347] Connected to a master server at 127.4.228.254:35243
I20260810 18:42:36.713151 17925 heartbeater.cc:464] Registering TS with master...
I20260810 18:42:36.713989 17925 heartbeater.cc:511] Master 127.4.228.254:35243 requested a full tablet report, sending...
I20260810 18:42:36.716835 17640 ts_manager.cc:194] Registered new tserver with Master: 674a0186f31945e7805c6b9d629d493b (127.4.228.193:41101)
I20260810 18:42:36.719375 17640 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.193@KRBTEST.COM'} at 127.4.228.193:34631
WARNING: no policy specified for kudu/127.4.228.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab.
Entry for principal kudu/127.4.228.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab.
I20260810 18:42:36.766631 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.194:0
--local_ip_for_outbound_sockets=127.4.228.194
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:35243
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:46853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:42:37.149276 17938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:42:37.149679 17938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:42:37.149890 17938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:42:37.157955 17938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:42:37.158143 17938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:42:37.158349 17938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.194
I20260810 18:42:37.168502 17938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:46853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:35243
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.194
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:42:37.170073 17938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:42:37.171619 17938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:42:37.190972 17947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:37.190909 17948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:37.190954 17951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:42:37.192219 17938 server_base.cc:1034] running on GCE node
I20260810 18:42:37.193143 17938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:42:37.194355 17938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:42:37.195601 17938 hybrid_clock.cc:648] HybridClock initialized: now 1786387357195530 us; error 63 us; skew 500 ppm
I20260810 18:42:37.196034 17938 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:42:37 dist-test-slave-7kzw krb5kdc[17573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387357, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:42:37.200212 17938 init.cc:377] Logged in from keytab as kudu/127.4.228.194@KRBTEST.COM (short username kudu)
I20260810 18:42:37.202013 17938 webserver.cc:460] Webserver started at http://127.4.228.194:40637/ using document root <none> and password file <none>
I20260810 18:42:37.202899 17938 fs_manager.cc:359] Metadata directory not provided
I20260810 18:42:37.203058 17938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:42:37.203406 17938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:42:37.207093 17938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-1/data/instance:
uuid: "0d4826fafcd64612a783b5378c31c7a2"
format_stamp: "Formatted at 2026-08-10 18:42:37 on dist-test-slave-7kzw"
server_key: "153938c8376d5c5bec100ae47e6dfe93"
server_key_iv: "dc673e5e9512adb106a42f3e6ddcbdc0"
server_key_version: "encryptionkey@0"
I20260810 18:42:37.208069 17938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance:
uuid: "0d4826fafcd64612a783b5378c31c7a2"
format_stamp: "Formatted at 2026-08-10 18:42:37 on dist-test-slave-7kzw"
server_key: "153938c8376d5c5bec100ae47e6dfe93"
server_key_iv: "dc673e5e9512adb106a42f3e6ddcbdc0"
server_key_version: "encryptionkey@0"
I20260810 18:42:37.214689 17938 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260810 18:42:37.219020 17960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:37.220665 17938 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260810 18:42:37.220888 17938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-1/wal
uuid: "0d4826fafcd64612a783b5378c31c7a2"
format_stamp: "Formatted at 2026-08-10 18:42:37 on dist-test-slave-7kzw"
server_key: "153938c8376d5c5bec100ae47e6dfe93"
server_key_iv: "dc673e5e9512adb106a42f3e6ddcbdc0"
server_key_version: "encryptionkey@0"
I20260810 18:42:37.221151 17938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:42:37.241609 17938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:42:37.246691 17938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:42:37.247088 17938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:42:37.248301 17938 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:42:37.249823 17938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:42:37.249902 17938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:37.250041 17938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:42:37.250092 17938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:37.286726 17938 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.194:38223
I20260810 18:42:37.286767 18125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.194:38223 every 8 connection(s)
I20260810 18:42:37.288297 17938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
I20260810 18:42:37.288883 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 17938
I20260810 18:42:37.289139 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance
I20260810 18:42:37.289772 5011 external_mini_cluster.cc:1428] Setting key 3f1312e21d477671c63a20ce5447d4b9
Aug 10 18:42:37 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387357, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:42:37.315675 18129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:37.291370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:36611 (local address 127.4.228.254:35243)
0810 18:42:37.291941 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:37.291948 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:37.293149 (+ 1201us) server_negotiation.cc:407] Connection header received
0810 18:42:37.296252 (+ 3103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:37.296256 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:37.296334 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:37.296440 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:37.298891 (+ 2451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:37.299661 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:37.302257 (+ 2596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:37.302811 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:37.302924 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:37.307534 (+ 4610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:37.307540 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:37.307543 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:37.307583 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:37.311200 (+ 3617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:37.312518 (+ 1318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:37.312522 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:37.312525 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:37.312718 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:37.313450 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:37.313454 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:37.313457 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:37.313825 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:37.313958 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:37.315119 (+ 1161us) server_negotiation.cc:299] Negotiation successful
0810 18:42:37.315351 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":318,"threads_started":1}
I20260810 18:42:37.317662 18130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:37.292076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.194:36611)
0810 18:42:37.292953 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:37.293016 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:37.295861 (+ 2845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:37.296763 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:37.296778 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:37.297694 (+ 916us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:37.298665 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:37.298699 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:37.299852 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:37.299860 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:37.302058 (+ 2198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:37.302079 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:37.303077 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:37.303088 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:37.303439 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:37.304251 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:37.304290 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:37.307196 (+ 2906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:37.311422 (+ 4226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:37.311440 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:37.311467 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:37.312328 (+ 861us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:37.312866 (+ 538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:37.312875 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:37.312883 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:37.313251 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:37.313985 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:37.313997 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:37.314607 (+ 610us) client_negotiation.cc:769] Sending connection context
0810 18:42:37.315011 (+ 404us) client_negotiation.cc:240] Negotiation successful
0810 18:42:37.315792 (+ 781us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":316,"threads_started":1}
I20260810 18:42:37.319754 18126 heartbeater.cc:347] Connected to a master server at 127.4.228.254:35243
I20260810 18:42:37.320163 18126 heartbeater.cc:464] Registering TS with master...
I20260810 18:42:37.321058 18126 heartbeater.cc:511] Master 127.4.228.254:35243 requested a full tablet report, sending...
I20260810 18:42:37.322959 17640 ts_manager.cc:194] Registered new tserver with Master: 0d4826fafcd64612a783b5378c31c7a2 (127.4.228.194:38223)
WARNING: no policy specified for kudu/127.4.228.195@KRBTEST.COM; defaulting to no policy
I20260810 18:42:37.323936 17640 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.194@KRBTEST.COM'} at 127.4.228.194:36611
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab.
Entry for principal kudu/127.4.228.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab.
I20260810 18:42:37.362923 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.195:0
--local_ip_for_outbound_sockets=127.4.228.195
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:35243
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:46853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:42:37.723166 17925 heartbeater.cc:503] Master 127.4.228.254:35243 was elected leader, sending a full tablet report...
W20260810 18:42:37.736317 18134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:42:37.736702 18134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:42:37.736878 18134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:42:37.744473 18134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:42:37.744666 18134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:42:37.744860 18134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.195
I20260810 18:42:37.755420 18134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:46853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:35243
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.195
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:42:37.757042 18134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:42:37.758587 18134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:42:37.773262 18139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:37.773183 18149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:37.773219 18141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:42:37.774289 18134 server_base.cc:1034] running on GCE node
I20260810 18:42:37.775074 18134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:42:37.776350 18134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:42:37.777629 18134 hybrid_clock.cc:648] HybridClock initialized: now 1786387357777546 us; error 69 us; skew 500 ppm
I20260810 18:42:37.778043 18134 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:42:37 dist-test-slave-7kzw krb5kdc[17573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387357, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:42:37.781804 18134 init.cc:377] Logged in from keytab as kudu/127.4.228.195@KRBTEST.COM (short username kudu)
I20260810 18:42:37.783617 18134 webserver.cc:460] Webserver started at http://127.4.228.195:38381/ using document root <none> and password file <none>
I20260810 18:42:37.784408 18134 fs_manager.cc:359] Metadata directory not provided
I20260810 18:42:37.784567 18134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:42:37.784933 18134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:42:37.788262 18134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-2/data/instance:
uuid: "996fd79a9e53449b8d81d725df80a198"
format_stamp: "Formatted at 2026-08-10 18:42:37 on dist-test-slave-7kzw"
server_key: "cfc8498184fc8ae9a60751cc711ae73d"
server_key_iv: "7f093c615cc5307e360296ec314a3540"
server_key_version: "encryptionkey@0"
I20260810 18:42:37.789191 18134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance:
uuid: "996fd79a9e53449b8d81d725df80a198"
format_stamp: "Formatted at 2026-08-10 18:42:37 on dist-test-slave-7kzw"
server_key: "cfc8498184fc8ae9a60751cc711ae73d"
server_key_iv: "7f093c615cc5307e360296ec314a3540"
server_key_version: "encryptionkey@0"
I20260810 18:42:37.795430 18134 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260810 18:42:37.799647 18161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:37.801218 18134 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:42:37.801438 18134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-2/wal
uuid: "996fd79a9e53449b8d81d725df80a198"
format_stamp: "Formatted at 2026-08-10 18:42:37 on dist-test-slave-7kzw"
server_key: "cfc8498184fc8ae9a60751cc711ae73d"
server_key_iv: "7f093c615cc5307e360296ec314a3540"
server_key_version: "encryptionkey@0"
I20260810 18:42:37.801687 18134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:42:37.818632 18134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:42:37.823757 18134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:42:37.824121 18134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:42:37.825296 18134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:42:37.827069 18134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:42:37.827201 18134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:37.827400 18134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:42:37.827507 18134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:37.867302 18134 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.195:45253
I20260810 18:42:37.867343 18330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.195:45253 every 8 connection(s)
I20260810 18:42:37.868870 18134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
I20260810 18:42:37.874408 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 18134
I20260810 18:42:37.874711 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance
I20260810 18:42:37.875182 5011 external_mini_cluster.cc:1428] Setting key e5e263abaed6a0c38c2d7be65b30cd17
Aug 10 18:42:37 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387357, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:42:37.891039 18337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:37.871814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:42221 (local address 127.4.228.254:35243)
0810 18:42:37.872315 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:37.872320 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:37.873275 (+ 955us) server_negotiation.cc:407] Connection header received
0810 18:42:37.875510 (+ 2235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:37.875516 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:37.875644 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:37.875776 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:37.877641 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:37.878554 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:37.880199 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:37.880619 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:37.880792 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:37.884720 (+ 3928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:37.884726 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:37.884729 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:37.884765 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:37.887573 (+ 2808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:37.888415 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:37.888422 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:37.888426 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:37.888578 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:37.889053 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:37.889058 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:37.889061 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:37.889362 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:37.889535 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:37.890317 (+ 782us) server_negotiation.cc:299] Negotiation successful
0810 18:42:37.890595 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":253,"threads_started":1}
I20260810 18:42:37.892201 18338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:37.872268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.195:42221)
0810 18:42:37.873036 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:37.873148 (+ 112us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:37.875160 (+ 2012us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:37.876048 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:37.876057 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:37.876782 (+ 725us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:37.877441 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:37.877461 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:37.878736 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:37.878740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:37.880015 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:37.880028 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:37.880901 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:37.880909 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:37.881178 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:37.881950 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:37.881977 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:37.884447 (+ 2470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:37.887782 (+ 3335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:37.887794 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:37.887814 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:37.888280 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:37.888703 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:42:37.888708 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:42:37.888712 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:42:37.888931 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:42:37.889500 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:37.889509 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:37.889914 (+ 405us) client_negotiation.cc:769] Sending connection context
0810 18:42:37.890214 (+ 300us) client_negotiation.cc:240] Negotiation successful
0810 18:42:37.890856 (+ 642us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":297,"threads_started":1}
I20260810 18:42:37.893811 18331 heartbeater.cc:347] Connected to a master server at 127.4.228.254:35243
I20260810 18:42:37.894164 18331 heartbeater.cc:464] Registering TS with master...
I20260810 18:42:37.895035 18331 heartbeater.cc:511] Master 127.4.228.254:35243 requested a full tablet report, sending...
I20260810 18:42:37.896729 17640 ts_manager.cc:194] Registered new tserver with Master: 996fd79a9e53449b8d81d725df80a198 (127.4.228.195:45253)
I20260810 18:42:37.897645 17640 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.195@KRBTEST.COM'} at 127.4.228.195:42221
I20260810 18:42:37.905167 5011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.4.228.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.228.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.228.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.193.keytab.
Entry for principal oryx/127.4.228.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.193.keytab.
WARNING: no policy specified for oryx/127.4.228.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.228.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.228.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.194.keytab.
Entry for principal oryx/127.4.228.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.194.keytab.
WARNING: no policy specified for oryx/127.4.228.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.228.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.228.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.195.keytab.
Entry for principal oryx/127.4.228.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.195.keytab.
I20260810 18:42:38.106117 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 17727
Aug 10 18:42:38 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.193@KRBTEST.COM
I20260810 18:42:38.126109 18353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:38.107555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47690 (local address 127.4.228.193:41101)
0810 18:42:38.108373 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:38.108381 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:38.108413 (+ 32us) server_negotiation.cc:407] Connection header received
0810 18:42:38.108490 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:38.108498 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:38.108792 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:38.108956 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:38.110670 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:38.111889 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:38.113359 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:38.114012 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:38.114131 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:38.119363 (+ 5232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:38.119380 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:38.119399 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:38.119452 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:38.123011 (+ 3559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:38.123833 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:38.123840 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:38.123843 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:38.123994 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:38.124395 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:38.124399 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:38.124401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:38.124801 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:38.125057 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:38.125503 (+ 446us) server_negotiation.cc:299] Negotiation successful
0810 18:42:38.125773 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":406,"threads_started":1}
I20260810 18:42:38.127825 17917 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:42:38.214812 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 17727
I20260810 18:42:38.245011 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 17938
Aug 10 18:42:38 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.194@KRBTEST.COM
I20260810 18:42:38.263230 18355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:38.246158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46346 (local address 127.4.228.194:38223)
0810 18:42:38.246869 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:38.246877 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:38.246903 (+ 26us) server_negotiation.cc:407] Connection header received
0810 18:42:38.246974 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:38.246981 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:38.247220 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:38.247442 (+ 222us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:38.248805 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:38.249937 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:38.251449 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:38.252021 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:38.252167 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:38.256890 (+ 4723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:38.256904 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:38.256922 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:38.256966 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:38.260114 (+ 3148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:38.260917 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:38.260925 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:38.260928 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:38.261065 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:38.261472 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:38.261477 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:38.261481 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:38.261863 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:38.262218 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:38.262544 (+ 326us) server_negotiation.cc:299] Negotiation successful
0810 18:42:38.262834 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":428,"threads_started":1}
I20260810 18:42:38.264788 18121 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:42:38.351051 18126 heartbeater.cc:503] Master 127.4.228.254:35243 was elected leader, sending a full tablet report...
I20260810 18:42:38.352535 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 17938
W20260810 18:42:38.379947 17621 connection.cc:535] server connection from 127.4.228.194:36611 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 18:42:38.380324 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 18134
Aug 10 18:42:38 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.195@KRBTEST.COM
I20260810 18:42:38.398339 18362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:38.381464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59548 (local address 127.4.228.195:45253)
0810 18:42:38.382022 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:38.382029 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:38.382059 (+ 30us) server_negotiation.cc:407] Connection header received
0810 18:42:38.382123 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:38.382131 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:38.382356 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:38.382554 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:38.383971 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:38.384952 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:38.386285 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:38.387040 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:38.387204 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:38.392033 (+ 4829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:38.392048 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:38.392065 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:38.392105 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:38.395317 (+ 3212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:38.396053 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:38.396061 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:38.396064 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:38.396196 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:38.396598 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:38.396604 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:38.396607 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:38.396989 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:38.397334 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:38.397668 (+ 334us) server_negotiation.cc:299] Negotiation successful
0810 18:42:38.397979 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":261,"threads_started":1}
I20260810 18:42:38.400167 18322 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:42:38.483846 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 18134
I20260810 18:42:38.508980 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 17590
I20260810 18:42:38.510562 17688 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:42:38.617264 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 17590
I20260810 18:42:38.651208 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:35243
--webserver_interface=127.4.228.254
--webserver_port=33435
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:46853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:35243
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:42:39.022962 18367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:42:39.023319 18367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:42:39.023414 18367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:42:39.030998 18367 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:42:39.031142 18367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:42:39.031191 18367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:42:39.031286 18367 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:42:39.031354 18367 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:42:39.042013 18367 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:46853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:35243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:35243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=33435
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:42:39.043552 18367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:42:39.045003 18367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:42:39.058701 18374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:39.058842 18375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:39.058799 18380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:42:39.059769 18367 server_base.cc:1034] running on GCE node
I20260810 18:42:39.060583 18367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:42:39.062165 18367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:42:39.063445 18367 hybrid_clock.cc:648] HybridClock initialized: now 1786387359063348 us; error 103 us; skew 500 ppm
I20260810 18:42:39.063913 18367 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:42:39 dist-test-slave-7kzw krb5kdc[17573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387359, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:42:39.067549 18367 init.cc:377] Logged in from keytab as kudu/127.4.228.254@KRBTEST.COM (short username kudu)
I20260810 18:42:39.069284 18367 webserver.cc:460] Webserver started at http://127.4.228.254:33435/ using document root <none> and password file <none>
I20260810 18:42:39.070148 18367 fs_manager.cc:359] Metadata directory not provided
I20260810 18:42:39.070354 18367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:42:39.076861 18367 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:42:39.080062 18389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:39.081490 18367 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:42:39.081738 18367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "df7a4c4084074982824c194343d2d7a8"
format_stamp: "Formatted at 2026-08-10 18:42:35 on dist-test-slave-7kzw"
server_key: "64173377726de776ae16f8f276d65cd1"
server_key_iv: "34c6db46f2cd59ce20da20b7936c2afc"
server_key_version: "encryptionkey@0"
I20260810 18:42:39.082922 18367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:42:39.112488 18367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:42:39.117756 18367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:42:39.118120 18367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:42:39.140834 18367 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:35243
I20260810 18:42:39.140889 18463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:35243 every 8 connection(s)
I20260810 18:42:39.142606 18367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:42:39.144289 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 18367
I20260810 18:42:39.145478 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.193:41101
--local_ip_for_outbound_sockets=127.4.228.193
--tserver_master_addrs=127.4.228.254:35243
--webserver_port=33317
--webserver_interface=127.4.228.193
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:46853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.4.228.193
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:42:39.154901 18464 sys_catalog.cc:263] Verifying existing consensus state
I20260810 18:42:39.159886 18464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8: Bootstrap starting.
I20260810 18:42:39.181879 18464 log.cc:824] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8: Log is configured to *not* fsync() on all Append() calls
I20260810 18:42:39.193065 18464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:42:39.193737 18464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8: Bootstrap complete.
I20260810 18:42:39.204538 18464 raft_consensus.cc:348] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df7a4c4084074982824c194343d2d7a8" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35243 } }
I20260810 18:42:39.205911 18464 raft_consensus.cc:729] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: df7a4c4084074982824c194343d2d7a8, State: Initialized, Role: FOLLOWER
I20260810 18:42:39.206748 18464 consensus_queue.cc:260] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df7a4c4084074982824c194343d2d7a8" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35243 } }
I20260810 18:42:39.207059 18464 raft_consensus.cc:388] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:42:39.207221 18464 raft_consensus.cc:482] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:42:39.207454 18464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8 [term 1 FOLLOWER]: Advancing to term 2
I20260810 18:42:39.212720 18464 raft_consensus.cc:504] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df7a4c4084074982824c194343d2d7a8" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35243 } }
I20260810 18:42:39.213379 18464 leader_election.cc:302] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df7a4c4084074982824c194343d2d7a8; no voters:
I20260810 18:42:39.214154 18464 leader_election.cc:288] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 18:42:39.214327 18469 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 18:42:39.215675 18469 raft_consensus.cc:686] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8 [term 2 LEADER]: Becoming Leader. State: Replica: df7a4c4084074982824c194343d2d7a8, State: Running, Role: LEADER
I20260810 18:42:39.216499 18469 consensus_queue.cc:237] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df7a4c4084074982824c194343d2d7a8" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35243 } }
I20260810 18:42:39.217114 18464 sys_catalog.cc:564] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:42:39.219748 18473 sys_catalog.cc:455] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df7a4c4084074982824c194343d2d7a8. Latest consensus state: current_term: 2 leader_uuid: "df7a4c4084074982824c194343d2d7a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df7a4c4084074982824c194343d2d7a8" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35243 } } }
I20260810 18:42:39.220178 18473 sys_catalog.cc:458] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8 [sys.catalog]: This master's current role is: LEADER
I20260810 18:42:39.220795 18470 sys_catalog.cc:455] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "df7a4c4084074982824c194343d2d7a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df7a4c4084074982824c194343d2d7a8" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35243 } } }
I20260810 18:42:39.221150 18470 sys_catalog.cc:458] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8 [sys.catalog]: This master's current role is: LEADER
I20260810 18:42:39.232319 18476 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:42:39.240948 18476 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:42:39.243136 18476 catalog_manager.cc:1259] Loaded cluster ID: c1182782702f4728b39064340fbbea3c
I20260810 18:42:39.243345 18476 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:42:39.248970 18476 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:42:39.251236 18476 catalog_manager.cc:5879] T 00000000000000000000000000000000 P df7a4c4084074982824c194343d2d7a8: Loaded TSK: 0
I20260810 18:42:39.253401 18476 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 18:42:39.567929 18467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:42:39.568257 18467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:42:39.568445 18467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:42:39.576509 18467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:42:39.576659 18467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:42:39.576864 18467 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.193
I20260810 18:42:39.588373 18467 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:46853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.193.keytab
--principal=oryx/127.4.228.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.193:41101
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.228.193
--webserver_port=33317
--tserver_master_addrs=127.4.228.254:35243
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.193
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:42:39.589922 18467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:42:39.591387 18467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:42:39.608425 18503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:39.608954 18502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:39.612438 18505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:42:39.612797 18467 server_base.cc:1034] running on GCE node
I20260810 18:42:39.613523 18467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:42:39.614766 18467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:42:39.616065 18467 hybrid_clock.cc:648] HybridClock initialized: now 1786387359615976 us; error 72 us; skew 500 ppm
I20260810 18:42:39.616506 18467 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:42:39 dist-test-slave-7kzw krb5kdc[17573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387359, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:42:39.620918 18467 init.cc:377] Logged in from keytab as oryx/127.4.228.193@KRBTEST.COM (short username oryx)
I20260810 18:42:39.623052 18467 webserver.cc:460] Webserver started at http://127.4.228.193:33317/ using document root <none> and password file <none>
I20260810 18:42:39.624380 18467 fs_manager.cc:359] Metadata directory not provided
I20260810 18:42:39.624594 18467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:42:39.631239 18467 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260810 18:42:39.634691 18516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:39.636467 18467 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260810 18:42:39.636718 18467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-0/wal
uuid: "674a0186f31945e7805c6b9d629d493b"
format_stamp: "Formatted at 2026-08-10 18:42:36 on dist-test-slave-7kzw"
server_key: "f9e9303f29883226380ae4542f57a96d"
server_key_iv: "93bdda39d1e2d5a8da6b071c90d56178"
server_key_version: "encryptionkey@0"
I20260810 18:42:39.637902 18467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:42:39.674233 18467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:42:39.679550 18467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:42:39.679922 18467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:42:39.681186 18467 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:42:39.682945 18467 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:42:39.683109 18467 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:39.683265 18467 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:42:39.683380 18467 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:39.725735 18467 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.193:41101
I20260810 18:42:39.725843 18687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.193:41101 every 8 connection(s)
I20260810 18:42:39.727421 18467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
I20260810 18:42:39.732125 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 18467
I20260810 18:42:39.732827 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.194:38223
--local_ip_for_outbound_sockets=127.4.228.194
--tserver_master_addrs=127.4.228.254:35243
--webserver_port=40637
--webserver_interface=127.4.228.194
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:46853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.4.228.194
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:42:39 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:39 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
W20260810 18:42:40.170409 18694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:42:40.170856 18694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:42:40.171025 18694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:42:40.179522 18694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:42:40.179744 18694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:42:40.179929 18694 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.194
I20260810 18:42:40.191026 18694 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:46853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.194.keytab
--principal=oryx/127.4.228.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.194:38223
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.228.194
--webserver_port=40637
--tserver_master_addrs=127.4.228.254:35243
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.194
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:42:40.192577 18694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:42:40.194065 18694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:42:40.215896 18701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:40.216150 18703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:40.215759 18705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:42:40.217924 18694 server_base.cc:1034] running on GCE node
I20260810 18:42:40.218832 18694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:42:40.219991 18694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:42:40.221704 18694 hybrid_clock.cc:648] HybridClock initialized: now 1786387360221613 us; error 91 us; skew 500 ppm
I20260810 18:42:40.222770 18694 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:42:40 dist-test-slave-7kzw krb5kdc[17573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387360, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:42:40.228417 18694 init.cc:377] Logged in from keytab as oryx/127.4.228.194@KRBTEST.COM (short username oryx)
I20260810 18:42:40.230659 18694 webserver.cc:460] Webserver started at http://127.4.228.194:40637/ using document root <none> and password file <none>
I20260810 18:42:40.231483 18694 fs_manager.cc:359] Metadata directory not provided
I20260810 18:42:40.231670 18694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:42:40.239125 18694 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.004s
I20260810 18:42:40.243432 18713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:40.245296 18694 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:42:40.245791 18694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-1/wal
uuid: "0d4826fafcd64612a783b5378c31c7a2"
format_stamp: "Formatted at 2026-08-10 18:42:37 on dist-test-slave-7kzw"
server_key: "153938c8376d5c5bec100ae47e6dfe93"
server_key_iv: "dc673e5e9512adb106a42f3e6ddcbdc0"
server_key_version: "encryptionkey@0"
I20260810 18:42:40.247136 18694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:42:40.272277 18694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:42:40.278231 18694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:42:40.278654 18694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:42:40.279879 18694 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:42:40.281538 18694 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:42:40.281689 18694 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:40.281840 18694 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:42:40.281936 18694 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:40.327502 18694 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.194:38223
I20260810 18:42:40.327540 18893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.194:38223 every 8 connection(s)
I20260810 18:42:40.329262 18694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
I20260810 18:42:40.333534 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 18694
I20260810 18:42:40.334575 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.195:45253
--local_ip_for_outbound_sockets=127.4.228.195
--tserver_master_addrs=127.4.228.254:35243
--webserver_port=38381
--webserver_interface=127.4.228.195
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:46853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.4.228.195
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:42:40 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:40 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
W20260810 18:42:40.993639 18690 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.245s user 0.001s sys 0.016s
W20260810 18:42:40.995887 18692 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:39.730886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:43523 (local address 127.4.228.254:35243)
0810 18:42:39.735072 (+ 4186us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:39.735095 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:39.735866 (+ 771us) server_negotiation.cc:407] Connection header received
0810 18:42:39.738259 (+ 2393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:39.738284 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:39.738929 (+ 645us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:39.739405 (+ 476us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:39.741145 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:39.743456 (+ 2311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:39.745442 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:39.746367 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:39.746631 (+ 264us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:40.995031 (+1248400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.193:43523: BlockingRecv error: recv got EOF from 127.4.228.193:43523 (error 108)
Metrics: {"server-negotiator.queue_time_us":3870,"thread_start_us":3648,"threads_started":1}
I20260810 18:42:40.995896 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:39.730958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.193:43523)
0810 18:42:39.735616 (+ 4658us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:39.735676 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:39.737564 (+ 1888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:39.739573 (+ 2009us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:39.739581 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:39.740271 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:39.740892 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:39.740912 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:39.743759 (+ 2847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:39.743765 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:39.745226 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:39.745244 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:39.746728 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:39.746741 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:39.747111 (+ 370us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:39.748047 (+ 936us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:39.748087 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:40.994604 (+1246517us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":4336,"thread_start_us":4141,"threads_started":1}
W20260810 18:42:40.997243 18688 heartbeater.cc:644] Failed to heartbeat to 127.4.228.254:35243 (0 consecutive failures): Not authorized: Failed to ping master at 127.4.228.254:35243: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Aug 10 18:42:41 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:41 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:41.014446 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:40.998020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.193:60315)
0810 18:42:40.998322 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:40.998343 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:40.998499 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:40.998884 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:40.998888 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:40.999282 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:40.999466 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:40.999475 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:41.000552 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:41.000564 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:41.001532 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:41.001544 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:41.002304 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:41.002310 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:41.002543 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:41.008977 (+ 6434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:41.009002 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:41.014069 (+ 5067us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166}
W20260810 18:42:41.014740 18692 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:40.998186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:60315 (local address 127.4.228.254:35243)
0810 18:42:40.998385 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:40.998393 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:40.998415 (+ 22us) server_negotiation.cc:407] Connection header received
0810 18:42:40.998662 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:40.998667 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:40.998733 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:40.998847 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:40.999624 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:41.000394 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:41.001694 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:41.002107 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:41.002270 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:41.014389 (+ 12119us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.193:60315: BlockingRecv error: recv got EOF from 127.4.228.193:60315 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 10 18:42:41 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:41 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:41.025835 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:41.016070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.193:43395)
0810 18:42:41.016275 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:41.016298 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:41.016431 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:41.016916 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:41.016923 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:41.017566 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:41.017756 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:41.017766 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:41.019184 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:41.019194 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:41.020212 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:41.020226 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:41.020960 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:41.020967 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:41.021193 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:41.021863 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:41.021883 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:41.025522 (+ 3639us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":69}
W20260810 18:42:41.026018 18692 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:41.016387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:43395 (local address 127.4.228.254:35243)
0810 18:42:41.016589 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:41.016595 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:41.016611 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:42:41.016676 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:41.016680 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:41.016743 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:41.016863 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:41.017930 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:41.019013 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:41.020374 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:41.020790 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:41.020938 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:41.025769 (+ 4831us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.193:43395: BlockingRecv error: recv got EOF from 127.4.228.193:43395 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260810 18:42:41.026437 18688 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 18:42:41.133466 18900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:42:41.133777 18900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:42:41.133920 18900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:42:41.141659 18900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:42:41.141824 18900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:42:41.142030 18900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.195
I20260810 18:42:41.153147 18900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:46853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/krb5kdc/oryx_127.4.228.195.keytab
--principal=oryx/127.4.228.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.195:45253
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.228.195
--webserver_port=38381
--tserver_master_addrs=127.4.228.254:35243
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.195
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:42:41.154728 18900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:42:41.156272 18900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:42:41.181725 18912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:42:41.181727 18900 server_base.cc:1034] running on GCE node
W20260810 18:42:41.181886 18915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:41.182411 18910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:42:41.183388 18900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:42:41.184656 18900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:42:41.185948 18900 hybrid_clock.cc:648] HybridClock initialized: now 1786387361185868 us; error 60 us; skew 500 ppm
I20260810 18:42:41.186391 18900 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:42:41 dist-test-slave-7kzw krb5kdc[17573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387361, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.228.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:42:41.190994 18900 init.cc:377] Logged in from keytab as oryx/127.4.228.195@KRBTEST.COM (short username oryx)
I20260810 18:42:41.193216 18900 webserver.cc:460] Webserver started at http://127.4.228.195:38381/ using document root <none> and password file <none>
I20260810 18:42:41.194108 18900 fs_manager.cc:359] Metadata directory not provided
I20260810 18:42:41.194306 18900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:42:41.201712 18900 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 18:42:41.205857 18928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:41.207901 18900 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 18:42:41.208159 18900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-2/wal
uuid: "996fd79a9e53449b8d81d725df80a198"
format_stamp: "Formatted at 2026-08-10 18:42:37 on dist-test-slave-7kzw"
server_key: "cfc8498184fc8ae9a60751cc711ae73d"
server_key_iv: "7f093c615cc5307e360296ec314a3540"
server_key_version: "encryptionkey@0"
I20260810 18:42:41.209267 18900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:42:41.231420 18900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:42:41.237829 18900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:42:41.238339 18900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:42:41.239689 18900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:42:41.241449 18900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:42:41.241585 18900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:41.241715 18900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:42:41.241837 18900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:41.299770 18900 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.195:45253
I20260810 18:42:41.299841 19098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.195:45253 every 8 connection(s)
I20260810 18:42:41.301517 18900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
I20260810 18:42:41.305562 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 18900
Aug 10 18:42:41 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.195@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:41 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.195@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:41.344435 19104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:41.306696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32792 (local address 127.4.228.254:35243)
0810 18:42:41.324129 (+ 17433us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:41.324142 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:41.324224 (+ 82us) server_negotiation.cc:407] Connection header received
0810 18:42:41.324333 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:41.324345 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:41.324533 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:41.324745 (+ 212us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:41.327120 (+ 2375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:41.328475 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:41.330383 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:41.331061 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:41.333252 (+ 2191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:41.335348 (+ 2096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:41.335359 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:41.335378 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:41.335416 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:41.339807 (+ 4391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:41.340794 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:41.340805 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:41.340817 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:41.341040 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:42:41.341542 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:42:41.341550 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:42:41.341555 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:42:41.342248 (+ 693us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:41.342619 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:41.343837 (+ 1218us) server_negotiation.cc:299] Negotiation successful
0810 18:42:41.344145 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17181,"thread_start_us":18386,"threads_started":1}
W20260810 18:42:41.645210 18898 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.287s user 0.005s sys 0.010s
I20260810 18:42:41.647085 18898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:40.333979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.194:38633)
0810 18:42:40.347596 (+ 13617us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:40.347679 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:40.350432 (+ 2753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:40.351616 (+ 1184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:40.351655 (+ 39us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:40.352515 (+ 860us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:40.353229 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:40.353252 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:40.354704 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:40.354709 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:40.356325 (+ 1616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:40.356340 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:40.357268 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:40.357278 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:40.357571 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:40.358422 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:40.358455 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:41.646042 (+1287587us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":13138,"thread_start_us":12801,"threads_started":1}
W20260810 18:42:41.647106 18897 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:40.334103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:38633 (local address 127.4.228.254:35243)
0810 18:42:40.347059 (+ 12956us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:40.347067 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:40.348715 (+ 1648us) server_negotiation.cc:407] Connection header received
0810 18:42:40.350923 (+ 2208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:40.350931 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:40.351055 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:40.351215 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:40.353474 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:40.354439 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:40.356502 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:40.356969 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:40.357131 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:41.646535 (+1289404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.194:38633: BlockingRecv error: recv got EOF from 127.4.228.194:38633 (error 108)
Metrics: {"server-negotiator.queue_time_us":12754,"thread_start_us":12565,"threads_started":1}
W20260810 18:42:41.648439 18894 heartbeater.cc:644] Failed to heartbeat to 127.4.228.254:35243 (0 consecutive failures): Not authorized: Failed to ping master at 127.4.228.254:35243: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Aug 10 18:42:41 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:41 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:41.661664 18898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:41.649323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.194:36523)
0810 18:42:41.649615 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:41.649646 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:41.649892 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:41.650325 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:41.650330 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:41.650912 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:41.651143 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:41.651171 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:41.652633 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:41.652638 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:41.653664 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:41.653678 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:41.654492 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:41.654499 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:41.654685 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:41.655536 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:41.655556 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:41.661193 (+ 5637us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":66}
W20260810 18:42:41.661790 18897 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:41.649483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:36523 (local address 127.4.228.254:35243)
0810 18:42:41.649704 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:41.649711 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:41.649735 (+ 24us) server_negotiation.cc:407] Connection header received
0810 18:42:41.650061 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:41.650067 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:41.650164 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:41.650290 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:41.651330 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:41.652476 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:41.653833 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:41.654272 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:41.654406 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:41.661473 (+ 7067us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.194:36523: BlockingRecv error: recv got EOF from 127.4.228.194:36523 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 10 18:42:41 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:41 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:41.678006 18898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:41.663200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.194:60921)
0810 18:42:41.664094 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:41.664122 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:41.664363 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:41.664895 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:41.664903 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:41.665528 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:41.665861 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:41.665880 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:41.668273 (+ 2393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:41.668282 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:41.670665 (+ 2383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:41.670690 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:41.671829 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:41.671840 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:41.672154 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:41.673011 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:41.673050 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:41.677680 (+ 4630us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":708}
W20260810 18:42:41.678237 18897 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:41.663897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:60921 (local address 127.4.228.254:35243)
0810 18:42:41.664098 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:41.664106 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:41.664222 (+ 116us) server_negotiation.cc:407] Connection header received
0810 18:42:41.664565 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:41.664573 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:41.664694 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:41.664814 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:41.666125 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:41.667948 (+ 1823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:41.670895 (+ 2947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:41.671588 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:41.671701 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:41.677937 (+ 6236us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.194:60921: BlockingRecv error: recv got EOF from 127.4.228.194:60921 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260810 18:42:41.678524 18894 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 10 18:42:42 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:42 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
W20260810 18:42:42.043728 18897 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:42.028050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:42671 (local address 127.4.228.254:35243)
0810 18:42:42.028291 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:42.028296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:42.029254 (+ 958us) server_negotiation.cc:407] Connection header received
0810 18:42:42.029561 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:42.029573 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:42.029752 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:42.029937 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:42.031619 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:42.033446 (+ 1827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:42.035000 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:42.036180 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:42.036386 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:42.043480 (+ 7094us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.193:42671: BlockingRecv error: recv got EOF from 127.4.228.193:42671 (error 108)
Metrics: {"server-negotiator.queue_time_us":112}
I20260810 18:42:42.043725 19110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:42.028004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.193:42671)
0810 18:42:42.029119 (+ 1115us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:42.029144 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:42.029335 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:42.030123 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:42.030129 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:42.030895 (+ 766us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:42.031368 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:42.031394 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:42.033727 (+ 2333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:42.033732 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:42.034803 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:42.034820 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:42.036476 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:42.036484 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:42.036722 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:42.037845 (+ 1123us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:42.037872 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:42.043111 (+ 5239us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":985,"thread_start_us":636,"threads_started":1}
W20260810 18:42:42.556191 19103 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.222s user 0.010s sys 0.020s
W20260810 18:42:42.557528 18692 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:41.305100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:33385 (local address 127.4.228.254:35243)
0810 18:42:41.306411 (+ 1311us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:41.306419 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:41.322574 (+ 16155us) server_negotiation.cc:407] Connection header received
0810 18:42:41.324883 (+ 2309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:41.324890 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:41.325008 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:41.325089 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:41.327802 (+ 2713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:41.328848 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:41.331236 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:41.331925 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:41.332036 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:42.557292 (+1225256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.195:33385: BlockingRecv error: recv got EOF from 127.4.228.195:33385 (error 108)
Metrics: {"server-negotiator.queue_time_us":1097}
I20260810 18:42:42.558135 19103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:41.306259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.195:33385)
0810 18:42:41.322314 (+ 16055us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:41.322406 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:41.324453 (+ 2047us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:41.325734 (+ 1281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:41.325751 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:41.326612 (+ 861us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:41.327539 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:41.327579 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:41.329070 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:41.329075 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:41.331076 (+ 2001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:41.331091 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:41.332450 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:41.332458 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:41.332811 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:41.333600 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:41.333665 (+ 65us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:42.557068 (+1223403us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":15608,"thread_start_us":15251,"threads_started":1}
W20260810 18:42:42.559404 19100 heartbeater.cc:644] Failed to heartbeat to 127.4.228.254:35243 (0 consecutive failures): Not authorized: Failed to ping master at 127.4.228.254:35243: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Aug 10 18:42:42 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.195@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:42 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.195@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:42.570541 19103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:42.560205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.195:56275)
0810 18:42:42.560446 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:42.560470 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:42.560616 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:42.561042 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:42.561046 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:42.561452 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:42.561624 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:42.561633 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:42.562841 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:42.562847 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:42.564338 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:42.564353 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:42.565342 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:42.565348 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:42.565483 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:42.566161 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:42.566181 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:42.570245 (+ 4064us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":80}
W20260810 18:42:42.570840 18692 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:42.560299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:56275 (local address 127.4.228.254:35243)
0810 18:42:42.560585 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:42.560589 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:42.560610 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:42:42.560750 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:42.560754 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:42.560818 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:42.560932 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:42.561769 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:42.562682 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:42.564575 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:42.565036 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:42.565147 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:42.570582 (+ 5435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.195:56275: BlockingRecv error: recv got EOF from 127.4.228.195:56275 (error 108)
Metrics: {"server-negotiator.queue_time_us":138}
Aug 10 18:42:42 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.195@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:42 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.195@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:42.581542 19103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:42.571559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.195:60533)
0810 18:42:42.571903 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:42.571925 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:42.572050 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:42.572408 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:42.572413 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:42.572757 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:42.572916 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:42.572925 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:42.574026 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:42.574031 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:42.575170 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:42.575188 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:42.576017 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:42.576023 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:42.576218 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:42.576849 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:42.576869 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:42.581220 (+ 4351us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":85}
W20260810 18:42:42.581727 18692 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:42.571738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:60533 (local address 127.4.228.254:35243)
0810 18:42:42.572044 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:42.572050 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:42.572070 (+ 20us) server_negotiation.cc:407] Connection header received
0810 18:42:42.572177 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:42.572183 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:42.572256 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:42.572365 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:42.573075 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:42.573870 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:42.575338 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:42.575833 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:42.575999 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:42.581492 (+ 5493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.195:60533: BlockingRecv error: recv got EOF from 127.4.228.195:60533 (error 108)
Metrics: {"server-negotiator.queue_time_us":169}
W20260810 18:42:42.581933 19100 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 10 18:42:42 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:42 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:42.690486 19111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:42.679318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.194:38519)
0810 18:42:42.679892 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:42.679911 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:42.680179 (+ 268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:42.680857 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:42.680862 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:42.681454 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:42.681611 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:42.681626 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:42.683004 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:42.683009 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:42.684023 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:42.684038 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:42.684870 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:42.684876 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:42.685012 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:42.685859 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:42.685879 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:42.690201 (+ 4322us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":348,"threads_started":1}
W20260810 18:42:42.690959 18692 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:42.679475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:38519 (local address 127.4.228.254:35243)
0810 18:42:42.679743 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:42.679750 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:42.680025 (+ 275us) server_negotiation.cc:407] Connection header received
0810 18:42:42.680409 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:42.680416 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:42.680563 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:42.680687 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:42.681870 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:42.682825 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:42.684224 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:42.684641 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:42.684761 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:42.690517 (+ 5756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.194:38519: BlockingRecv error: recv got EOF from 127.4.228.194:38519 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 10 18:42:43 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:43 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:43.055097 19112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:43.044858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.193:38157)
0810 18:42:43.045506 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:43.045542 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:43.045706 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:43.046236 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:43.046241 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:43.046853 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:43.047049 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:43.047058 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:43.048191 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:43.048196 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:43.049137 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:43.049150 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:43.049886 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:43.049891 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:43.050044 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:43.050955 (+ 911us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:43.050974 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:43.054820 (+ 3846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":407,"threads_started":1}
W20260810 18:42:43.055398 18692 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:43.045022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:38157 (local address 127.4.228.254:35243)
0810 18:42:43.045278 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:43.045287 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:43.045634 (+ 347us) server_negotiation.cc:407] Connection header received
0810 18:42:43.045915 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:43.045919 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:43.046019 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:43.046160 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:43.047289 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:43.048037 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:43.049284 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:43.049706 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:43.049808 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:43.055066 (+ 5258us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.193:38157: BlockingRecv error: recv got EOF from 127.4.228.193:38157 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 10 18:42:43 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.195@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:43 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.195@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:43.593097 19113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:43.582685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.195:53811)
0810 18:42:43.583268 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:43.583285 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:43.583407 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:43.583762 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:43.583766 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:43.584238 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:43.584389 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:43.584398 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:43.585470 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:43.585474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:43.586556 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:43.586571 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:43.587479 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:43.587485 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:43.587655 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:43.588389 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:43.588409 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:43.592835 (+ 4426us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":328,"threads_started":1}
W20260810 18:42:43.593372 19114 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:43.582686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:53811 (local address 127.4.228.254:35243)
0810 18:42:43.583281 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:43.583285 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:43.583373 (+ 88us) server_negotiation.cc:407] Connection header received
0810 18:42:43.583513 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:43.583517 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:43.583579 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:43.583673 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:43.584535 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:43.585320 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:43.586735 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:43.587218 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:43.587331 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:43.593092 (+ 5761us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.195:53811: BlockingRecv error: recv got EOF from 127.4.228.195:53811 (error 108)
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":314,"threads_started":1}
Aug 10 18:42:43 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:43 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:43.703872 19115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:43.691761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.194:42389)
0810 18:42:43.692435 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:43.692452 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:43.692704 (+ 252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:43.693223 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:43.693231 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:43.693861 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:43.694118 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:43.694137 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:43.695575 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:43.695580 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:43.696542 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:43.696556 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:43.697479 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:43.697484 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:43.697665 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:43.698387 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:43.698408 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:43.703548 (+ 5140us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":387,"threads_started":1}
W20260810 18:42:43.704332 19114 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:43.691963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:42389 (local address 127.4.228.254:35243)
0810 18:42:43.692278 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:43.692285 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:43.692597 (+ 312us) server_negotiation.cc:407] Connection header received
0810 18:42:43.692862 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:43.692867 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:43.692982 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:43.693109 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:43.694332 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:43.695423 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:43.696760 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:43.697233 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:43.697354 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:43.704051 (+ 6697us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.194:42389: BlockingRecv error: recv got EOF from 127.4.228.194:42389 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Aug 10 18:42:44 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:44 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:44.067324 19120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:44.056658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.193:50261)
0810 18:42:44.057279 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:44.057302 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:44.057432 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:44.058173 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:44.058179 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:44.058702 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:44.058952 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:44.058968 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:44.060240 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:44.060244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:44.061206 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:44.061219 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:44.062056 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:44.062061 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:44.062210 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:44.062956 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:44.062976 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:44.067011 (+ 4035us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":325,"threads_started":1}
W20260810 18:42:44.067612 19114 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:44.056863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:50261 (local address 127.4.228.254:35243)
0810 18:42:44.057107 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:44.057116 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:44.057409 (+ 293us) server_negotiation.cc:407] Connection header received
0810 18:42:44.057855 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:44.057864 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:44.057952 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:44.058073 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:44.059143 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:44.060048 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:44.061380 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:44.061861 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:44.061969 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:44.067315 (+ 5346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.193:50261: BlockingRecv error: recv got EOF from 127.4.228.193:50261 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 10 18:42:44 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.195@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:44 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.195@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:44.605858 19121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:44.594456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.195:39531)
0810 18:42:44.595202 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:44.595224 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:44.595353 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:44.596043 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:44.596049 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:44.596638 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:44.596963 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:44.596981 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:44.598566 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:44.598571 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:44.599614 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:44.599627 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:44.600427 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:44.600433 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:44.600652 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:44.601315 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:44.601337 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:44.605432 (+ 4095us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":344,"threads_started":1}
W20260810 18:42:44.605989 19123 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:44.594726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:39531 (local address 127.4.228.254:35243)
0810 18:42:44.595299 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:44.595304 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:44.595324 (+ 20us) server_negotiation.cc:407] Connection header received
0810 18:42:44.595628 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:44.595637 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:44.595761 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:44.595915 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:44.597206 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:44.598330 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:44.599791 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:44.600243 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:44.600398 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:44.605694 (+ 5296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.195:39531: BlockingRecv error: recv got EOF from 127.4.228.195:39531 (error 108)
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":307,"threads_started":1}
Aug 10 18:42:44 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:44 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:44.716240 19124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:44.705062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.194:51149)
0810 18:42:44.705668 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:44.705697 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:44.705851 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:44.706243 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:44.706248 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:44.706754 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:44.706932 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:44.706943 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:44.708262 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:44.708267 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:44.709372 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:44.709388 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:44.710241 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:44.710247 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:44.710662 (+ 415us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:44.711426 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:44.711446 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:44.715987 (+ 4541us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":354,"threads_started":1}
W20260810 18:42:44.716590 19123 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:44.705246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:51149 (local address 127.4.228.254:35243)
0810 18:42:44.705439 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:44.705444 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:44.705827 (+ 383us) server_negotiation.cc:407] Connection header received
0810 18:42:44.705971 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:44.705976 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:44.706042 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:44.706135 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:44.707152 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:44.708048 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:44.709608 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:44.710064 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:44.710176 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:44.716284 (+ 6108us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.194:51149: BlockingRecv error: recv got EOF from 127.4.228.194:51149 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 10 18:42:45 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:45 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:45.079272 19126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:45.068519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.193:39073)
0810 18:42:45.069049 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:45.069070 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:45.069229 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:45.069612 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:45.069616 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:45.070013 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:45.070194 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:45.070204 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:45.071568 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:45.071577 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:45.073053 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:45.073071 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:45.073943 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:45.073949 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:45.074151 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:45.074971 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:45.074990 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:45.079005 (+ 4015us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":310,"threads_started":1}
W20260810 18:42:45.079504 19123 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:45.068602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:39073 (local address 127.4.228.254:35243)
0810 18:42:45.068858 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:45.068865 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:45.069225 (+ 360us) server_negotiation.cc:407] Connection header received
0810 18:42:45.069338 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:45.069342 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:45.069430 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:45.069530 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:45.070366 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:45.071415 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:45.073321 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:45.073777 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:45.073927 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:45.079237 (+ 5310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.193:39073: BlockingRecv error: recv got EOF from 127.4.228.193:39073 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 10 18:42:45 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.195@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:45 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.195@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:45.620846 19127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:45.607475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.195:36365)
0810 18:42:45.609418 (+ 1943us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:45.609449 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:45.609605 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:45.610080 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:45.610086 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:45.610659 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:45.610846 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:45.610857 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:45.612115 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:45.612119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:45.613086 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:45.613100 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:45.614087 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:45.614093 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:45.614299 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:45.615095 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:45.615115 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:45.620436 (+ 5321us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1807,"thread_start_us":1657,"threads_started":1}
W20260810 18:42:45.621170 19128 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:45.607621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:36365 (local address 127.4.228.254:35243)
0810 18:42:45.609092 (+ 1471us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:45.609099 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:45.609563 (+ 464us) server_negotiation.cc:407] Connection header received
0810 18:42:45.609741 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:45.609747 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:45.609867 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:45.609960 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:45.611030 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:45.611897 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:45.613301 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:45.613864 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:45.614065 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:45.620788 (+ 6723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.195:36365: BlockingRecv error: recv got EOF from 127.4.228.195:36365 (error 108)
Metrics: {"server-negotiator.queue_time_us":1335,"thread_start_us":1191,"threads_started":1}
Aug 10 18:42:45 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:45 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:45.727790 19129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:45.717355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.194:40071)
0810 18:42:45.717995 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:45.718014 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:45.718164 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:45.718702 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:45.718707 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:45.719112 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:45.719297 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:45.719308 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:45.720560 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:45.720565 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:45.721607 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:45.721622 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:45.722377 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:45.722382 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:45.722619 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:45.723298 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:45.723319 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:45.727249 (+ 3930us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":362,"threads_started":1}
W20260810 18:42:45.727880 19128 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:45.717539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:40071 (local address 127.4.228.254:35243)
0810 18:42:45.717832 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:45.717839 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:45.718120 (+ 281us) server_negotiation.cc:407] Connection header received
0810 18:42:45.718355 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:45.718361 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:45.718434 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:45.718647 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:45.719502 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:45.720395 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:45.721787 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:45.722214 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:45.722322 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:45.727583 (+ 5261us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.194:40071: BlockingRecv error: recv got EOF from 127.4.228.194:40071 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 10 18:42:46 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:46 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:46.096873 19130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:46.080549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.193:49983)
0810 18:42:46.081213 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:46.081235 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:46.081422 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:46.081849 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:46.081855 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:46.082379 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:46.082632 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:46.082643 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:46.084133 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:46.084143 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:46.085210 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:46.085223 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:46.086054 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:46.086062 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:46.086270 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:46.092081 (+ 5811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:46.092105 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:46.096476 (+ 4371us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":365,"threads_started":1}
W20260810 18:42:46.097101 19128 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:46.080699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:49983 (local address 127.4.228.254:35243)
0810 18:42:46.081008 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:46.081015 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:46.081338 (+ 323us) server_negotiation.cc:407] Connection header received
0810 18:42:46.081609 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:46.081615 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:46.081688 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:46.081814 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:46.082811 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:46.083983 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:46.085379 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:46.085869 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:46.086020 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:46.096785 (+ 10765us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.193:49983: BlockingRecv error: recv got EOF from 127.4.228.193:49983 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 10 18:42:46 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.195@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:46 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.195@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:46.633402 19132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:46.622192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.195:38569)
0810 18:42:46.623142 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:46.623161 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:46.623292 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:46.624021 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:46.624028 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:46.624537 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:46.624732 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:46.624743 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:46.625841 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:46.625845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:46.627122 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:46.627138 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:46.627936 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:46.627950 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:46.628101 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:46.628815 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:46.628835 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:46.633118 (+ 4283us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":833,"thread_start_us":411,"threads_started":1}
W20260810 18:42:46.633692 19133 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:46.622601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:38569 (local address 127.4.228.254:35243)
0810 18:42:46.623669 (+ 1068us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:46.623674 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:46.623696 (+ 22us) server_negotiation.cc:407] Connection header received
0810 18:42:46.623751 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:46.623756 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:46.623841 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:46.623938 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:46.624894 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:46.625689 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:46.627322 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:46.627767 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:46.627878 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:46.633388 (+ 5510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.195:38569: BlockingRecv error: recv got EOF from 127.4.228.195:38569 (error 108)
Metrics: {"server-negotiator.queue_time_us":965,"thread_start_us":828,"threads_started":1}
Aug 10 18:42:46 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:46 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:46.739204 19135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:46.729080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.194:38773)
0810 18:42:46.729785 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:46.729803 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:46.729967 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:46.730332 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:46.730337 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:46.730839 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:46.730992 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:46.731002 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:46.732035 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:46.732051 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:46.732994 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:46.733007 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:46.733727 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:46.733733 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:46.733925 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:46.734659 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:46.734702 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:46.738727 (+ 4025us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":440,"threads_started":1}
W20260810 18:42:46.739269 19133 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:46.729310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:38773 (local address 127.4.228.254:35243)
0810 18:42:46.729524 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:46.729533 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:46.729899 (+ 366us) server_negotiation.cc:407] Connection header received
0810 18:42:46.730111 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:46.730123 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:46.730186 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:46.730296 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:46.731160 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:46.731889 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:46.733158 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:46.733569 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:46.733710 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:46.739008 (+ 5298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.194:38773: BlockingRecv error: recv got EOF from 127.4.228.194:38773 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 10 18:42:47 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:47 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:47.108726 19136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:47.098091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.193:35599)
0810 18:42:47.098717 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:47.098746 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:47.098879 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:47.099291 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:47.099294 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:47.099742 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:47.099927 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:47.099936 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:47.101051 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:47.101055 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:47.102001 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:47.102014 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:47.102857 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:47.102863 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:47.103009 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:47.103750 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:47.103769 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:47.108438 (+ 4669us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":390,"threads_started":1}
W20260810 18:42:47.108991 19133 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:47.098226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:35599 (local address 127.4.228.254:35243)
0810 18:42:47.098412 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:47.098417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:47.098847 (+ 430us) server_negotiation.cc:407] Connection header received
0810 18:42:47.098999 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:47.099003 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:47.099091 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:47.099206 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:47.100095 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:47.100890 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:47.102204 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:47.102670 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:47.102779 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:47.108720 (+ 5941us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.193:35599: BlockingRecv error: recv got EOF from 127.4.228.193:35599 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 10 18:42:47 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.195@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:47 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.195@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:47.646404 19137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:47.634616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.195:56925)
0810 18:42:47.635271 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:47.635291 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:47.635431 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:47.635866 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:47.635870 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:47.636340 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:47.636506 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:47.636515 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:47.637746 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:47.637752 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:47.639067 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:47.639083 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:47.640691 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:47.640698 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:47.640943 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:47.641677 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:47.641697 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:47.646036 (+ 4339us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":305,"threads_started":1}
W20260810 18:42:47.646699 19139 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:47.634729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:56925 (local address 127.4.228.254:35243)
0810 18:42:47.635310 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:47.635315 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:47.635379 (+ 64us) server_negotiation.cc:407] Connection header received
0810 18:42:47.635555 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:47.635560 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:47.635624 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:47.635758 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:47.636777 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:47.637552 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:47.639735 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:47.640492 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:47.640635 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:47.646355 (+ 5720us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.195:56925: BlockingRecv error: recv got EOF from 127.4.228.195:56925 (error 108)
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":331,"threads_started":1}
Aug 10 18:42:47 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:47 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:47.751737 19141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:47.740866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.194:59101)
0810 18:42:47.741728 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:47.741749 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:47.741903 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:47.742372 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:47.742377 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:47.743046 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:47.743304 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:47.743316 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:47.744446 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:47.744450 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:47.745395 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:47.745408 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:47.746232 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:47.746236 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:47.746364 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:47.747097 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:47.747116 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:47.751439 (+ 4323us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":730,"thread_start_us":605,"threads_started":1}
W20260810 18:42:47.752043 19139 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:47.740984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:59101 (local address 127.4.228.254:35243)
0810 18:42:47.741205 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:47.741211 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:47.741870 (+ 659us) server_negotiation.cc:407] Connection header received
0810 18:42:47.742075 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:47.742082 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:47.742188 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:47.742291 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:47.743468 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:47.744294 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:47.745571 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:47.746028 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:47.746166 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:47.751721 (+ 5555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.194:59101: BlockingRecv error: recv got EOF from 127.4.228.194:59101 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 10 18:42:48 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:48 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:48.121477 19143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:48.110002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.193:40857)
0810 18:42:48.110734 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:48.110758 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:48.110903 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:48.111306 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:48.111310 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:48.111755 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:48.111970 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:48.111979 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:48.113424 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:48.113435 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:48.114655 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:48.114684 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:48.115545 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:48.115550 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:48.115691 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:48.116490 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:48.116509 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:48.121115 (+ 4606us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":366,"threads_started":1}
W20260810 18:42:48.121656 19139 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:48.110067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:40857 (local address 127.4.228.254:35243)
0810 18:42:48.110330 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:48.110341 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:48.110850 (+ 509us) server_negotiation.cc:407] Connection header received
0810 18:42:48.111015 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:48.111020 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:48.111086 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:48.111201 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:48.112171 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:48.113254 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:48.114870 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:48.115348 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:48.115457 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:48.121362 (+ 5905us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.193:40857: BlockingRecv error: recv got EOF from 127.4.228.193:40857 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 10 18:42:48 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.195@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:48 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.195@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:48.659277 19144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:48.647602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.195:51657)
0810 18:42:48.648209 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:48.648228 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:48.648439 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:48.649128 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:48.649133 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:48.649767 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:48.650043 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:48.650058 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:48.651796 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:48.651801 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:48.652979 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:48.652996 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:48.653866 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:48.653871 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:48.654021 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:48.654797 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:48.654818 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:48.659001 (+ 4183us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":293,"threads_started":1}
W20260810 18:42:48.659832 19146 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:48.647801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:51657 (local address 127.4.228.254:35243)
0810 18:42:48.648630 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:48.648634 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:48.648651 (+ 17us) server_negotiation.cc:407] Connection header received
0810 18:42:48.648762 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:48.648767 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:48.648850 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:48.649003 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:48.650315 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:48.651555 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:48.653155 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:48.653676 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:48.653790 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:48.659307 (+ 5517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.195:51657: BlockingRecv error: recv got EOF from 127.4.228.195:51657 (error 108)
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":571,"threads_started":1}
Aug 10 18:42:48 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:48 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:48.763619 19147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:48.752961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.194:36121)
0810 18:42:48.753715 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:48.753738 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:48.753895 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:48.754263 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:48.754267 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:48.754742 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:48.754934 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:48.754944 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:48.756132 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:48.756137 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:48.757351 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:48.757367 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:48.758191 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:48.758196 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:48.758327 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:48.759125 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:48.759168 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:48.763352 (+ 4184us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":367,"threads_started":1}
W20260810 18:42:48.763902 19146 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:48.753068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:36121 (local address 127.4.228.254:35243)
0810 18:42:48.753321 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:48.753328 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:48.753873 (+ 545us) server_negotiation.cc:407] Connection header received
0810 18:42:48.754008 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:48.754013 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:48.754082 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:48.754183 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:48.755096 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:48.755953 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:48.757503 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:48.757987 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:48.758100 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:48.763601 (+ 5501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.194:36121: BlockingRecv error: recv got EOF from 127.4.228.194:36121 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Aug 10 18:42:49 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:49 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:49.133539 19149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:49.122839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.193:50637)
0810 18:42:49.123504 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:49.123532 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:49.123710 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:49.124206 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:49.124212 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:49.124727 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:49.124920 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:49.124930 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:49.126134 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:49.126140 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:49.127293 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:49.127307 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:49.128176 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:49.128183 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:49.128380 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:49.129067 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:49.129113 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:49.133149 (+ 4036us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":328,"threads_started":1}
W20260810 18:42:49.133701 19146 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:49.123032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:50637 (local address 127.4.228.254:35243)
0810 18:42:49.123267 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:49.123275 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:49.123688 (+ 413us) server_negotiation.cc:407] Connection header received
0810 18:42:49.123890 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:49.123897 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:49.124003 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:49.124129 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:49.125093 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:49.125937 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:49.127463 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:49.127942 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:49.128110 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:49.133413 (+ 5303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.193:50637: BlockingRecv error: recv got EOF from 127.4.228.193:50637 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 10 18:42:49 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.195@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:49 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.195@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:49.673846 19150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:49.660354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.195:40395)
0810 18:42:49.661098 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:49.661123 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:49.661334 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:49.661834 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:49.661839 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:49.662581 (+ 742us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:49.662881 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:49.662916 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:49.664938 (+ 2022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:49.664945 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:49.666914 (+ 1969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:49.666938 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:49.668220 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:49.668226 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:49.668434 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:49.669239 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:49.669282 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:49.673313 (+ 4031us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":415,"threads_started":1}
W20260810 18:42:49.674034 19152 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:49.660544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:40395 (local address 127.4.228.254:35243)
0810 18:42:49.661136 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:49.661143 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:49.661252 (+ 109us) server_negotiation.cc:407] Connection header received
0810 18:42:49.661492 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:49.661500 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:49.661624 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:49.661769 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:49.663103 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:49.664695 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:49.667133 (+ 2438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:49.667999 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:49.668180 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:49.673608 (+ 5428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.195:40395: BlockingRecv error: recv got EOF from 127.4.228.195:40395 (error 108)
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":314,"threads_started":1}
Aug 10 18:42:49 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:49 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:49.775852 19153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:49.764799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.194:43249)
0810 18:42:49.765492 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:49.765512 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:49.765736 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:49.766195 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:49.766200 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:49.766724 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:49.766914 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:49.766923 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:49.768381 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:49.768391 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:49.769457 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:49.769471 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:49.770370 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:49.770378 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:49.770629 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:49.771323 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:49.771344 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:49.775441 (+ 4097us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":373,"threads_started":1}
W20260810 18:42:49.775913 19152 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:49.764993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:43249 (local address 127.4.228.254:35243)
0810 18:42:49.765296 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:49.765303 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:49.765650 (+ 347us) server_negotiation.cc:407] Connection header received
0810 18:42:49.765892 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:49.765899 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:49.765985 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:49.766115 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:49.767094 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:49.768198 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:49.769644 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:49.770146 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:49.770311 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:49.775679 (+ 5368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.194:43249: BlockingRecv error: recv got EOF from 127.4.228.194:43249 (error 108)
Metrics: {"server-negotiator.queue_time_us":119}
Aug 10 18:42:50 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:50 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:50.148548 19154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:50.134640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.193:43311)
0810 18:42:50.135253 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:50.135282 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:50.135428 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:50.135820 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:50.135825 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:50.136316 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:50.136498 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:50.136509 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:50.137827 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:50.137833 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:50.139523 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:50.139541 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:50.140769 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:50.140775 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:50.140926 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:50.141813 (+ 887us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:50.141841 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:50.147896 (+ 6055us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":280,"threads_started":1}
W20260810 18:42:50.148739 19152 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:50.134916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:43311 (local address 127.4.228.254:35243)
0810 18:42:50.135184 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:50.135189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:50.135403 (+ 214us) server_negotiation.cc:407] Connection header received
0810 18:42:50.135548 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:50.135552 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:50.135632 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:50.135726 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:50.136760 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:50.137635 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:50.139768 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:50.140510 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:50.140666 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:50.148366 (+ 7700us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.193:43311: BlockingRecv error: recv got EOF from 127.4.228.193:43311 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 10 18:42:50 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.195@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:50 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.195@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:50.687506 19155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:50.675571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.195:48635)
0810 18:42:50.676304 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:50.676324 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:50.676481 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:50.677058 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:50.677062 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:50.677485 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:50.677667 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:50.677677 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:50.679276 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:50.679282 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:50.680794 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:50.680814 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:50.681808 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:50.681814 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:50.681951 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:50.682736 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:50.682758 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:50.687186 (+ 4428us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":373,"threads_started":1}
W20260810 18:42:50.687836 19156 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:50.675824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:48635 (local address 127.4.228.254:35243)
0810 18:42:50.676700 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:50.676705 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:50.676724 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:42:50.676784 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:50.676788 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:50.676868 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:50.676970 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:50.677849 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:50.679099 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:50.680986 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:50.681564 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:50.681683 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:50.687545 (+ 5862us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.195:48635: BlockingRecv error: recv got EOF from 127.4.228.195:48635 (error 108)
Metrics: {"server-negotiator.queue_time_us":748,"thread_start_us":611,"threads_started":1}
Aug 10 18:42:50 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:50 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:50.789760 19157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:50.777236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.194:52315)
0810 18:42:50.777916 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:50.777938 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:50.778128 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:50.778745 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:50.778752 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:50.779543 (+ 791us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:50.779896 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:50.779910 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:50.781127 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:50.781136 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:50.782240 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:50.782254 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:50.783212 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:50.783219 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:50.783378 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:50.784351 (+ 973us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:50.784382 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:50.789357 (+ 4975us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":387,"threads_started":1}
W20260810 18:42:50.789928 19156 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:50.777370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:52315 (local address 127.4.228.254:35243)
0810 18:42:50.777616 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:50.777626 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:50.778050 (+ 424us) server_negotiation.cc:407] Connection header received
0810 18:42:50.778327 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:50.778332 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:50.778405 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:50.778598 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:50.780085 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:50.780948 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:50.782412 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:50.782997 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:50.783133 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:50.789611 (+ 6478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.194:52315: BlockingRecv error: recv got EOF from 127.4.228.194:52315 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 10 18:42:51 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:51 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:51.165287 19158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:51.149965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.193:33457)
0810 18:42:51.150567 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:51.150590 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:51.150735 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:51.151139 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:51.151143 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:51.151709 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:51.151887 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:51.151897 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:51.153365 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:51.153371 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:51.154375 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:51.154388 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:51.155309 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:51.155314 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:51.155446 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:51.160443 (+ 4997us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:51.160467 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:51.164803 (+ 4336us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":281,"threads_started":1}
W20260810 18:42:51.165354 19156 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:51.150079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:33457 (local address 127.4.228.254:35243)
0810 18:42:51.150360 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:51.150368 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:51.150707 (+ 339us) server_negotiation.cc:407] Connection header received
0810 18:42:51.150893 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:51.150898 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:51.150967 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:51.151058 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:51.152075 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:51.153149 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:51.154642 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:51.155112 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:51.155230 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:51.165059 (+ 9829us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.193:33457: BlockingRecv error: recv got EOF from 127.4.228.193:33457 (error 108)
Metrics: {"server-negotiator.queue_time_us":121}
Aug 10 18:42:51 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.195@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:51 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.195@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:51.704569 19161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:51.688790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.195:56387)
0810 18:42:51.689565 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:51.689582 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:51.689716 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:51.690041 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:51.690045 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:51.690460 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:51.690703 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:51.690713 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:51.691862 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:51.691867 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:51.692891 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:51.692904 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:51.693729 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:51.693734 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:51.693868 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:51.699609 (+ 5741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:51.699729 (+ 120us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:51.704305 (+ 4576us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":316,"threads_started":1}
W20260810 18:42:51.704958 19162 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:51.688830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:56387 (local address 127.4.228.254:35243)
0810 18:42:51.689540 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:51.689546 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:51.689648 (+ 102us) server_negotiation.cc:407] Connection header received
0810 18:42:51.689807 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:51.689811 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:51.689889 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:51.689988 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:51.690848 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:51.691684 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:51.693070 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:51.693523 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:51.693637 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:51.704597 (+ 10960us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.195:56387: BlockingRecv error: recv got EOF from 127.4.228.195:56387 (error 108)
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":459,"threads_started":1}
Aug 10 18:42:51 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:51 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:51.805361 19163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:51.791155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.194:52565)
0810 18:42:51.791763 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:51.791786 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:51.791939 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:51.792589 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:51.792789 (+ 200us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:51.793292 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:51.793535 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:51.793555 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:51.795358 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:51.795366 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:51.797221 (+ 1855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:51.797241 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:51.798465 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:51.798496 (+ 31us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:51.798853 (+ 357us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:51.799662 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:51.799692 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:51.804923 (+ 5231us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":290,"threads_started":1}
W20260810 18:42:51.805460 19162 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:51.791241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:52565 (local address 127.4.228.254:35243)
0810 18:42:51.791501 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:51.791508 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:51.791869 (+ 361us) server_negotiation.cc:407] Connection header received
0810 18:42:51.792108 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:51.792129 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:51.792271 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:51.792395 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:51.793737 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:51.795139 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:51.797425 (+ 2286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:51.798217 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:51.798393 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:51.805182 (+ 6789us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.194:52565: BlockingRecv error: recv got EOF from 127.4.228.194:52565 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 10 18:42:52 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:52 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:52.177435 19164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:52.166710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.193:35603)
0810 18:42:52.167366 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:52.167392 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:52.167585 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:52.167959 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:52.167964 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:52.168468 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:52.168669 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:52.168679 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:52.169856 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:52.169864 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:52.170987 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:52.171000 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:52.171790 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:52.171797 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:52.172002 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:52.172740 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:52.172762 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:52.177175 (+ 4413us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":382,"threads_started":1}
W20260810 18:42:52.177711 19162 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:52.166890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:35603 (local address 127.4.228.254:35243)
0810 18:42:52.167148 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:52.167155 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:52.167501 (+ 346us) server_negotiation.cc:407] Connection header received
0810 18:42:52.167725 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:52.167733 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:52.167809 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:52.167920 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:52.168833 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:52.169700 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:52.171155 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:52.171619 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:52.171772 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:52.177393 (+ 5621us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.193:35603: BlockingRecv error: recv got EOF from 127.4.228.193:35603 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 10 18:42:52 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.195@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:52 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.195@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:52.717517 19165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:52.705947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.195:48897)
0810 18:42:52.706641 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:52.706669 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:52.706845 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:52.707451 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:52.707457 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:52.708057 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:52.708329 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:52.708351 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:52.709821 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:52.709826 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:52.711023 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:52.711038 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:52.711853 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:52.711859 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:52.711990 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:52.712753 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:52.712774 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:52.717229 (+ 4455us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":368,"threads_started":1}
W20260810 18:42:52.717890 19166 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:52.706169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:48897 (local address 127.4.228.254:35243)
0810 18:42:52.706920 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:52.706928 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:52.706948 (+ 20us) server_negotiation.cc:407] Connection header received
0810 18:42:52.707006 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:52.707013 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:52.707197 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:52.707349 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:52.708528 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:52.709640 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:52.711210 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:52.711694 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:52.711808 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:52.717546 (+ 5738us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.195:48897: BlockingRecv error: recv got EOF from 127.4.228.195:48897 (error 108)
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":479,"threads_started":1}
Aug 10 18:42:52 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:52 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:52.818044 19167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:52.806875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.194:50645)
0810 18:42:52.807448 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:52.807466 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:52.807600 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:52.807982 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:52.807987 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:52.808486 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:52.808728 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:52.808740 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:52.810099 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:52.810104 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:52.811430 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:52.811451 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:52.812252 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:52.812258 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:52.812408 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:52.813135 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:52.813161 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:52.817654 (+ 4493us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":314,"threads_started":1}
W20260810 18:42:52.818369 19166 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:52.807083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:50645 (local address 127.4.228.254:35243)
0810 18:42:52.807358 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:52.807366 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:52.807627 (+ 261us) server_negotiation.cc:407] Connection header received
0810 18:42:52.807705 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:52.807710 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:52.807782 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:52.807885 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:52.808936 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:52.809919 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:52.811614 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:52.812062 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:52.812193 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:52.817910 (+ 5717us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.194:50645: BlockingRecv error: recv got EOF from 127.4.228.194:50645 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 10 18:42:53 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:53 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:53.190011 19168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:53.178732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.193:50661)
0810 18:42:53.179385 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:53.179408 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:53.179540 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:53.179997 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:53.180002 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:53.180429 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:53.180680 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:53.180692 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:53.181844 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:53.181849 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:53.183036 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:53.183053 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:53.184253 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:53.184283 (+ 30us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:53.184529 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:53.185398 (+ 869us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:53.185421 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:53.189744 (+ 4323us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":380,"threads_started":1}
W20260810 18:42:53.190411 19166 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:53.178799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:50661 (local address 127.4.228.254:35243)
0810 18:42:53.179041 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:53.179046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:53.179531 (+ 485us) server_negotiation.cc:407] Connection header received
0810 18:42:53.179665 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:53.179670 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:53.179772 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:53.179878 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:53.180879 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:53.181658 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:53.183262 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:53.183996 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:53.184148 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:53.190049 (+ 5901us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.193:50661: BlockingRecv error: recv got EOF from 127.4.228.193:50661 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 10 18:42:53 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.195@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:53 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.195@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:53.729718 19169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:53.718824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.195:53643)
0810 18:42:53.719348 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:53.719368 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:53.719495 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:53.720037 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:53.720041 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:53.720500 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:53.720683 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:53.720693 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:53.721868 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:53.721873 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:53.723163 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:53.723180 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:53.724220 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:53.724229 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:53.724442 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:53.725319 (+ 877us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:53.725342 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:53.729480 (+ 4138us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":275,"threads_started":1}
W20260810 18:42:53.730130 19172 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:53.718913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:53643 (local address 127.4.228.254:35243)
0810 18:42:53.719690 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:53.719695 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:53.719712 (+ 17us) server_negotiation.cc:407] Connection header received
0810 18:42:53.719766 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:53.719770 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:53.719842 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:53.719938 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:53.720900 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:53.721699 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:53.723334 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:53.723988 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:53.724134 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:53.729775 (+ 5641us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.195:53643: BlockingRecv error: recv got EOF from 127.4.228.195:53643 (error 108)
Metrics: {"server-negotiator.queue_time_us":676,"thread_start_us":566,"threads_started":1}
Aug 10 18:42:53 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:53 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:53.829530 19173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:53.819069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.194:32967)
0810 18:42:53.819666 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:53.819685 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:53.819830 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:53.820196 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:53.820200 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:53.820658 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:53.820833 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:53.820842 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:53.821938 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:53.821943 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:53.822952 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:53.822966 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:53.823848 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:53.823854 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:53.824005 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:53.824668 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:53.824696 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:53.829246 (+ 4550us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":315,"threads_started":1}
W20260810 18:42:53.829708 19172 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:53.819181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:32967 (local address 127.4.228.254:35243)
0810 18:42:53.819456 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:53.819463 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:53.819830 (+ 367us) server_negotiation.cc:407] Connection header received
0810 18:42:53.819947 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:53.819952 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:53.820025 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:53.820145 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:53.821008 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:53.821791 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:53.823130 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:53.823607 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:53.823761 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:53.829463 (+ 5702us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.194:32967: BlockingRecv error: recv got EOF from 127.4.228.194:32967 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
Aug 10 18:42:54 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:54 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:54.202457 19174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:54.191150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.193:58377)
0810 18:42:54.191659 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:54.191682 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:54.191810 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:54.192183 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:54.192187 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:54.192640 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:54.192799 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:54.192809 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:54.194618 (+ 1809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:54.194623 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:54.195669 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:54.195684 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:54.196620 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:54.196625 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:54.196793 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:54.197601 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:54.197625 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:54.202099 (+ 4474us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":280,"threads_started":1}
W20260810 18:42:54.202728 19172 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:54.191157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:58377 (local address 127.4.228.254:35243)
0810 18:42:54.191354 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:54.191359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:54.191794 (+ 435us) server_negotiation.cc:407] Connection header received
0810 18:42:54.191905 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:54.191909 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:54.191978 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:54.192073 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:54.192966 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:54.194403 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:54.195846 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:54.196409 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:54.196540 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:54.202376 (+ 5836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.193:58377: BlockingRecv error: recv got EOF from 127.4.228.193:58377 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 10 18:42:54 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.195@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:54 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.195@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:54.741816 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:54.731084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.195:59507)
0810 18:42:54.731718 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:54.731738 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:54.731867 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:54.732356 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:54.732362 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:54.732841 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:54.733020 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:54.733030 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:54.734197 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:54.734202 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:54.735374 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:54.735390 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:54.736287 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:54.736293 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:54.736465 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:54.737242 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:54.737264 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:54.741465 (+ 4201us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":351,"threads_started":1}
W20260810 18:42:54.742090 19176 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:54.731131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:59507 (local address 127.4.228.254:35243)
0810 18:42:54.731721 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:54.731726 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:54.731855 (+ 129us) server_negotiation.cc:407] Connection header received
0810 18:42:54.732029 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:54.732033 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:54.732152 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:54.732257 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:54.733219 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:54.733993 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:54.735559 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:54.736051 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:54.736163 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:54.741789 (+ 5626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.195:59507: BlockingRecv error: recv got EOF from 127.4.228.195:59507 (error 108)
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":310,"threads_started":1}
Aug 10 18:42:54 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:54 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:54.841416 19177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:54.830963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.194:33245)
0810 18:42:54.831581 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:54.831619 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:54.831786 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:54.832310 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:54.832316 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:54.833079 (+ 763us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:54.833291 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:54.833302 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:54.834460 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:54.834518 (+ 58us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:54.835474 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:54.835487 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:54.836247 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:54.836252 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:54.836375 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:54.837058 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:54.837078 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:54.841038 (+ 3960us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":287,"threads_started":1}
W20260810 18:42:54.841538 19176 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:54.831406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:33245 (local address 127.4.228.254:35243)
0810 18:42:54.831625 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:54.831644 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:54.831704 (+ 60us) server_negotiation.cc:407] Connection header received
0810 18:42:54.831929 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:54.831935 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:54.832024 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:54.832138 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:54.833433 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:54.834261 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:54.835645 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:54.836107 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:54.836218 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:54.841284 (+ 5066us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.194:33245: BlockingRecv error: recv got EOF from 127.4.228.194:33245 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 10 18:42:55 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:55 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:55.214968 19179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:55.204482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.193:39197)
0810 18:42:55.205392 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:55.205423 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:55.205591 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:55.206050 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:55.206058 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:55.206534 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:55.206772 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:55.206782 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:55.208024 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:55.208045 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:55.209136 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:55.209151 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:55.209921 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:55.209926 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:55.210091 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:55.210778 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:55.210797 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:55.214650 (+ 3853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":348,"threads_started":1}
W20260810 18:42:55.215260 19176 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:55.204745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:39197 (local address 127.4.228.254:35243)
0810 18:42:55.205011 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:55.205018 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:55.205551 (+ 533us) server_negotiation.cc:407] Connection header received
0810 18:42:55.205736 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:55.205742 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:55.205843 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:55.205937 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:55.206944 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:55.207847 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:55.209300 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:55.209759 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:55.209868 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:55.214930 (+ 5062us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.193:39197: BlockingRecv error: recv got EOF from 127.4.228.193:39197 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 10 18:42:55 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.195@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:55 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.195@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:55.753680 19180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:55.743155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.195:37325)
0810 18:42:55.743843 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:55.743868 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:55.744033 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:55.744466 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:55.744473 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:55.745109 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:55.745269 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:55.745278 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:55.746843 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:55.746847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:55.747790 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:55.747802 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:55.748570 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:55.748575 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:55.748729 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:55.749436 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:55.749456 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:55.753422 (+ 3966us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":388,"threads_started":1}
W20260810 18:42:55.754169 19181 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:55.743244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:37325 (local address 127.4.228.254:35243)
0810 18:42:55.744041 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:55.744046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:55.744070 (+ 24us) server_negotiation.cc:407] Connection header received
0810 18:42:55.744142 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:55.744149 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:55.744265 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:55.744392 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:55.745445 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:55.746691 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:55.747950 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:55.748406 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:55.748513 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:55.753787 (+ 5274us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.195:37325: BlockingRecv error: recv got EOF from 127.4.228.195:37325 (error 108)
Metrics: {"server-negotiator.queue_time_us":687,"thread_start_us":457,"threads_started":1}
Aug 10 18:42:55 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:55 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.194@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:55.853550 19184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:55.842999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.194:45745)
0810 18:42:55.843548 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:55.843565 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:55.843750 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:55.844164 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:55.844169 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:55.844603 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:55.844805 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:55.844818 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:55.846105 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:55.846110 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:55.847299 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:55.847314 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:55.848271 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:55.848279 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:55.848516 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:55.849171 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:55.849190 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:55.853266 (+ 4076us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":295,"threads_started":1}
W20260810 18:42:55.853826 19181 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:55.843205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:45745 (local address 127.4.228.254:35243)
0810 18:42:55.843374 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:55.843379 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:55.843685 (+ 306us) server_negotiation.cc:407] Connection header received
0810 18:42:55.843901 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:55.843908 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:55.843992 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:55.844085 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:55.845050 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:55.845880 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:55.847453 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:55.848089 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:55.848208 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:55.853550 (+ 5342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.194:45745: BlockingRecv error: recv got EOF from 127.4.228.194:45745 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 10 18:42:56 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:42:56 dist-test-slave-7kzw krb5kdc[17573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.228.193@KRBTEST.COM for oryx/127.4.228.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:42:56.231106 19185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:56.216092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35243 (local address 127.4.228.193:55435)
0810 18:42:56.216642 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:56.216663 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:56.216825 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:56.217209 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:56.217214 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:56.217754 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:56.217935 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:56.217944 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:56.219106 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:56.219113 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:56.220060 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:56.220071 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:56.220790 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:56.220796 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:56.220966 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:56.226512 (+ 5546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:42:56.226593 (+ 81us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:56.230725 (+ 4132us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.228.254:35243: Server oryx/127.4.228.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":293,"threads_started":1}
W20260810 18:42:56.231290 19181 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:42:56.216172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:55435 (local address 127.4.228.254:35243)
0810 18:42:56.216379 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:56.216387 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:56.216752 (+ 365us) server_negotiation.cc:407] Connection header received
0810 18:42:56.216958 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:56.216963 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:56.217038 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:56.217184 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:56.218076 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:56.218973 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:56.220207 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:56.220638 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:56.220770 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:56.230968 (+ 10198us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.228.193:55435: BlockingRecv error: recv got EOF from 127.4.228.193:55435 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 18:42:56.314284 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 18467
I20260810 18:42:56.329524 19188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:56.315951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37158 (local address 127.4.228.193:41101)
0810 18:42:56.316746 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:56.316753 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:56.316784 (+ 31us) server_negotiation.cc:407] Connection header received
0810 18:42:56.316848 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:56.316854 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:56.317105 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:56.317281 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:56.319380 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:56.320514 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:56.321828 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:56.322525 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:56.322676 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:56.325571 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:56.325617 (+ 46us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:56.325635 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:56.325684 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:56.328726 (+ 3042us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.4.228.193@KRBTEST.COM not found in keytab (ticket kvno 2)
0810 18:42:56.329085 (+ 359us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37158: Request ticket server kudu/127.4.228.193@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":436,"threads_started":1}
W20260810 18:42:56.329660 19188 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37158: Request ticket server kudu/127.4.228.193@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260810 18:42:56.329587 5011 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 18467: Not authorized: Client connection negotiation failed: client connection to 127.4.228.193:41101: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.4.228.193@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260810 18:42:56.329845 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 18467
I20260810 18:42:56.367705 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 18694
I20260810 18:42:56.381542 19189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:56.368845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44212 (local address 127.4.228.194:38223)
0810 18:42:56.369485 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:56.369494 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:56.369522 (+ 28us) server_negotiation.cc:407] Connection header received
0810 18:42:56.369593 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:56.369600 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:56.369860 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:56.370039 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:56.372081 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:56.373176 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:56.374458 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:56.375164 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:56.375279 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:56.377926 (+ 2647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:56.377935 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:56.377951 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:56.378010 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:56.380975 (+ 2965us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.4.228.194@KRBTEST.COM not found in keytab (ticket kvno 2)
0810 18:42:56.381303 (+ 328us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44212: Request ticket server kudu/127.4.228.194@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":359,"threads_started":1}
W20260810 18:42:56.381686 19189 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44212: Request ticket server kudu/127.4.228.194@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260810 18:42:56.381773 5011 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 18694: Not authorized: Client connection negotiation failed: client connection to 127.4.228.194:38223: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.4.228.194@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260810 18:42:56.381927 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 18694
I20260810 18:42:56.410039 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 18900
I20260810 18:42:56.424078 19191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:56.411367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52056 (local address 127.4.228.195:45253)
0810 18:42:56.412007 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:56.412015 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:56.412043 (+ 28us) server_negotiation.cc:407] Connection header received
0810 18:42:56.412114 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:56.412132 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:56.412394 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:56.412637 (+ 243us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:56.414281 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:56.415463 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:56.416728 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:56.417393 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:56.417537 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:56.420405 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:56.420416 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:56.420432 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:42:56.420478 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:56.423297 (+ 2819us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.4.228.195@KRBTEST.COM not found in keytab (ticket kvno 2)
0810 18:42:56.423696 (+ 399us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52056: Request ticket server kudu/127.4.228.195@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":312,"threads_started":1}
W20260810 18:42:56.424161 5011 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 18900: Not authorized: Client connection negotiation failed: client connection to 127.4.228.195:45253: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.4.228.195@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260810 18:42:56.424216 19191 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52056: Request ticket server kudu/127.4.228.195@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260810 18:42:56.424343 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 18900
I20260810 18:42:56.452375 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 18367
I20260810 18:42:56.454167 18459 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:42:56.575420 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 18367
2026-08-10T18:42:56Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (22845 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-10T18:42:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:42:56Z Disabled control of system clock
I20260810 18:42:56.681293 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:35833
--webserver_interface=127.4.228.254
--webserver_port=0
--builtin_ntp_servers=127.4.228.212:32791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:35833
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 18:42:57.056699 19197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:42:57.057086 19197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:42:57.057175 19197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:42:57.066226 19197 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:42:57.066392 19197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:42:57.066438 19197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:42:57.066584 19197 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:42:57.066646 19197 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:42:57.077313 19197 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:32791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:35833
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:35833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:42:57.078861 19197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:42:57.080806 19197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:42:57.093966 19204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:57.094162 19207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:57.093940 19205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:42:57.095039 19197 server_base.cc:1034] running on GCE node
I20260810 18:42:57.095928 19197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:42:57.097579 19197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:42:57.098840 19197 hybrid_clock.cc:648] HybridClock initialized: now 1786387377098757 us; error 57 us; skew 500 ppm
I20260810 18:42:57.099283 19197 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:42:57.102041 19197 webserver.cc:460] Webserver started at http://127.4.228.254:37551/ using document root <none> and password file <none>
I20260810 18:42:57.102977 19197 fs_manager.cc:359] Metadata directory not provided
I20260810 18:42:57.103082 19197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:42:57.103492 19197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:42:57.106837 19197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "7d59406c24cb4a1abfc68debbfac3b2b"
format_stamp: "Formatted at 2026-08-10 18:42:57 on dist-test-slave-7kzw"
server_key: "962ed799820c6f80703c14b783dddcd9"
server_key_iv: "15257c5416a34ba32db14a8cd5ee1e6d"
server_key_version: "encryptionkey@0"
I20260810 18:42:57.107762 19197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "7d59406c24cb4a1abfc68debbfac3b2b"
format_stamp: "Formatted at 2026-08-10 18:42:57 on dist-test-slave-7kzw"
server_key: "962ed799820c6f80703c14b783dddcd9"
server_key_iv: "15257c5416a34ba32db14a8cd5ee1e6d"
server_key_version: "encryptionkey@0"
I20260810 18:42:57.113672 19197 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 18:42:57.117766 19220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:57.119562 19197 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260810 18:42:57.119817 19197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "7d59406c24cb4a1abfc68debbfac3b2b"
format_stamp: "Formatted at 2026-08-10 18:42:57 on dist-test-slave-7kzw"
server_key: "962ed799820c6f80703c14b783dddcd9"
server_key_iv: "15257c5416a34ba32db14a8cd5ee1e6d"
server_key_version: "encryptionkey@0"
I20260810 18:42:57.120090 19197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:42:57.154497 19197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:42:57.155565 19197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:42:57.155933 19197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:42:57.177544 19197 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:35833
I20260810 18:42:57.177565 19304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:35833 every 8 connection(s)
I20260810 18:42:57.179224 19197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:42:57.183559 19305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:42:57.184690 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 19197
I20260810 18:42:57.184947 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:42:57.185848 5011 external_mini_cluster.cc:1428] Setting key bc04fdb3a82645aa5a163e9da9f7f6f3
I20260810 18:42:57.200860 19305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d59406c24cb4a1abfc68debbfac3b2b: Bootstrap starting.
I20260810 18:42:57.203363 19309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:57.188544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44882 (local address 127.4.228.254:35833)
0810 18:42:57.189845 (+ 1301us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:57.189866 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:57.189937 (+ 71us) server_negotiation.cc:407] Connection header received
0810 18:42:57.191769 (+ 1832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:57.191803 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:57.192529 (+ 726us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:57.193040 (+ 511us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:57.193610 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:57.195510 (+ 1900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:57.197114 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:57.198332 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:57.198556 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:57.199991 (+ 1435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:57.200005 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:57.200028 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:42:57.200849 (+ 821us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:57.200963 (+ 114us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:42:57.201004 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:57.201154 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:57.201513 (+ 359us) server_negotiation.cc:299] Negotiation successful
0810 18:42:57.201887 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":726,"thread_start_us":383,"threads_started":1}
I20260810 18:42:57.205328 19305 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d59406c24cb4a1abfc68debbfac3b2b: Neither blocks nor log segments found. Creating new log.
I20260810 18:42:57.206576 19305 log.cc:824] T 00000000000000000000000000000000 P 7d59406c24cb4a1abfc68debbfac3b2b: Log is configured to *not* fsync() on all Append() calls
I20260810 18:42:57.210005 19305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d59406c24cb4a1abfc68debbfac3b2b: No bootstrap required, opened a new log
I20260810 18:42:57.218252 19305 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d59406c24cb4a1abfc68debbfac3b2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d59406c24cb4a1abfc68debbfac3b2b" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35833 } }
I20260810 18:42:57.218780 19305 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d59406c24cb4a1abfc68debbfac3b2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:42:57.218897 19305 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d59406c24cb4a1abfc68debbfac3b2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d59406c24cb4a1abfc68debbfac3b2b, State: Initialized, Role: FOLLOWER
I20260810 18:42:57.219584 19305 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d59406c24cb4a1abfc68debbfac3b2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d59406c24cb4a1abfc68debbfac3b2b" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35833 } }
I20260810 18:42:57.219921 19305 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d59406c24cb4a1abfc68debbfac3b2b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:42:57.220122 19305 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d59406c24cb4a1abfc68debbfac3b2b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:42:57.220359 19305 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d59406c24cb4a1abfc68debbfac3b2b [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:42:57.222756 19305 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d59406c24cb4a1abfc68debbfac3b2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d59406c24cb4a1abfc68debbfac3b2b" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35833 } }
I20260810 18:42:57.223335 19305 leader_election.cc:302] T 00000000000000000000000000000000 P 7d59406c24cb4a1abfc68debbfac3b2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d59406c24cb4a1abfc68debbfac3b2b; no voters:
I20260810 18:42:57.224068 19305 leader_election.cc:288] T 00000000000000000000000000000000 P 7d59406c24cb4a1abfc68debbfac3b2b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:42:57.224261 19313 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d59406c24cb4a1abfc68debbfac3b2b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:42:57.225654 19313 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d59406c24cb4a1abfc68debbfac3b2b [term 1 LEADER]: Becoming Leader. State: Replica: 7d59406c24cb4a1abfc68debbfac3b2b, State: Running, Role: LEADER
I20260810 18:42:57.226495 19313 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d59406c24cb4a1abfc68debbfac3b2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d59406c24cb4a1abfc68debbfac3b2b" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35833 } }
I20260810 18:42:57.226694 19305 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d59406c24cb4a1abfc68debbfac3b2b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:42:57.230289 19317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d59406c24cb4a1abfc68debbfac3b2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d59406c24cb4a1abfc68debbfac3b2b. Latest consensus state: current_term: 1 leader_uuid: "7d59406c24cb4a1abfc68debbfac3b2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d59406c24cb4a1abfc68debbfac3b2b" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35833 } } }
I20260810 18:42:57.230895 19317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d59406c24cb4a1abfc68debbfac3b2b [sys.catalog]: This master's current role is: LEADER
I20260810 18:42:57.231479 19314 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d59406c24cb4a1abfc68debbfac3b2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d59406c24cb4a1abfc68debbfac3b2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d59406c24cb4a1abfc68debbfac3b2b" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35833 } } }
I20260810 18:42:57.231927 19314 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d59406c24cb4a1abfc68debbfac3b2b [sys.catalog]: This master's current role is: LEADER
I20260810 18:42:57.239665 19329 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:42:57.244549 19329 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:42:57.254832 19329 catalog_manager.cc:1347] Generated new cluster ID: 55239de9a6a74a71a94dcb86ecf645cc
I20260810 18:42:57.255061 19329 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:42:57.286109 19329 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:42:57.287768 19329 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:42:57.301715 19329 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d59406c24cb4a1abfc68debbfac3b2b: Generated new TSK 0
I20260810 18:42:57.302867 19329 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:42:57.316645 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.193:0
--local_ip_for_outbound_sockets=127.4.228.193
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:35833
--builtin_ntp_servers=127.4.228.212:32791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260810 18:42:58.506896 19344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:42:58.508183 19344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:42:58.508807 19344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:42:58.532642 19344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:42:58.532783 19344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:42:58.532948 19344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.193
I20260810 18:42:58.548913 19344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:32791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:35833
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.193
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:42:58.550451 19344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:42:58.555964 19344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:42:58.610172 19357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:58.610152 19351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:58.610682 19350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:42:58.615219 19344 server_base.cc:1034] running on GCE node
I20260810 18:42:58.617992 19344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:42:58.624593 19344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:42:58.633271 19344 hybrid_clock.cc:648] HybridClock initialized: now 1786387378632671 us; error 571 us; skew 500 ppm
I20260810 18:42:58.635208 19344 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:42:58.645780 19344 webserver.cc:460] Webserver started at http://127.4.228.193:38129/ using document root <none> and password file <none>
I20260810 18:42:58.649253 19344 fs_manager.cc:359] Metadata directory not provided
I20260810 18:42:58.649597 19344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:42:58.651235 19344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:42:58.662320 19344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/data/instance:
uuid: "71c3f194f3934857aac82a811f6db154"
format_stamp: "Formatted at 2026-08-10 18:42:58 on dist-test-slave-7kzw"
server_key: "cb6ccdc0b35b67ca8fe82fcb0e41222b"
server_key_iv: "52697c4e89774913401ae9effae18fbf"
server_key_version: "encryptionkey@0"
I20260810 18:42:58.666286 19344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance:
uuid: "71c3f194f3934857aac82a811f6db154"
format_stamp: "Formatted at 2026-08-10 18:42:58 on dist-test-slave-7kzw"
server_key: "cb6ccdc0b35b67ca8fe82fcb0e41222b"
server_key_iv: "52697c4e89774913401ae9effae18fbf"
server_key_version: "encryptionkey@0"
I20260810 18:42:58.689049 19344 fs_manager.cc:691] Time spent creating directory manager: real 0.021s user 0.009s sys 0.008s
I20260810 18:42:58.707166 19366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:58.713372 19344 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.007s sys 0.001s
I20260810 18:42:58.714242 19344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/wal
uuid: "71c3f194f3934857aac82a811f6db154"
format_stamp: "Formatted at 2026-08-10 18:42:58 on dist-test-slave-7kzw"
server_key: "cb6ccdc0b35b67ca8fe82fcb0e41222b"
server_key_iv: "52697c4e89774913401ae9effae18fbf"
server_key_version: "encryptionkey@0"
I20260810 18:42:58.715217 19344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:42:58.768625 19344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:42:58.773046 19344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:42:58.774890 19344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:42:58.779699 19344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:42:58.785974 19344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:42:58.786454 19344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:58.786921 19344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:42:58.787299 19344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:58.931143 19344 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.193:40517
I20260810 18:42:58.931655 19527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.193:40517 every 8 connection(s)
I20260810 18:42:58.936391 19344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
I20260810 18:42:58.947690 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 19344
I20260810 18:42:58.947969 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance
I20260810 18:42:58.949237 5011 external_mini_cluster.cc:1428] Setting key e146e7ea99714de0a5c205e1246b0801
I20260810 18:42:58.964071 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.194:0
--local_ip_for_outbound_sockets=127.4.228.194
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:35833
--builtin_ntp_servers=127.4.228.212:32791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 18:42:58.998574 19532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:58.945540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:50515 (local address 127.4.228.254:35833)
0810 18:42:58.947625 (+ 2085us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:58.947632 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:58.953921 (+ 6289us) server_negotiation.cc:407] Connection header received
0810 18:42:58.967036 (+ 13115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:58.967085 (+ 49us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:58.967920 (+ 835us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:58.968683 (+ 763us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:58.973752 (+ 5069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:58.979316 (+ 5564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:58.985904 (+ 6588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:58.987339 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:58.987653 (+ 314us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:58.993402 (+ 5749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:58.993429 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:58.993441 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:42:58.993500 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:58.993635 (+ 135us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:42:58.993705 (+ 70us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:58.994134 (+ 429us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:58.995274 (+ 1140us) server_negotiation.cc:299] Negotiation successful
0810 18:42:58.995866 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1539,"thread_start_us":1117,"threads_started":1}
I20260810 18:42:59.001219 19533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:58.948086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35833 (local address 127.4.228.193:50515)
0810 18:42:58.952898 (+ 4812us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:58.953300 (+ 402us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:58.965500 (+ 12200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:58.969623 (+ 4123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:58.969660 (+ 37us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:58.970104 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:58.972833 (+ 2729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:58.972919 (+ 86us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:58.980160 (+ 7241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:58.980182 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:58.985237 (+ 5055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:58.985318 (+ 81us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:58.988208 (+ 2890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:58.988264 (+ 56us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:58.989385 (+ 1121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:58.991691 (+ 2306us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:42:58.991709 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:58.991894 (+ 185us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:42:58.992345 (+ 451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:58.994138 (+ 1793us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:58.994175 (+ 37us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:58.994254 (+ 79us) client_negotiation.cc:769] Sending connection context
0810 18:42:58.995119 (+ 865us) client_negotiation.cc:240] Negotiation successful
0810 18:42:58.997651 (+ 2532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1748,"thread_start_us":1149,"threads_started":1}
I20260810 18:42:59.003458 19528 heartbeater.cc:347] Connected to a master server at 127.4.228.254:35833
I20260810 18:42:59.003873 19528 heartbeater.cc:464] Registering TS with master...
I20260810 18:42:59.004917 19528 heartbeater.cc:511] Master 127.4.228.254:35833 requested a full tablet report, sending...
I20260810 18:42:59.007676 19251 ts_manager.cc:194] Registered new tserver with Master: 71c3f194f3934857aac82a811f6db154 (127.4.228.193:40517)
I20260810 18:42:59.009465 19251 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.228.193:50515
W20260810 18:42:59.374768 19535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:42:59.375090 19535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:42:59.375249 19535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:42:59.382725 19535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:42:59.382845 19535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:42:59.383021 19535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.194
I20260810 18:42:59.392705 19535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:32791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:35833
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.194
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:42:59.394187 19535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:42:59.396214 19535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:42:59.411569 19541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:59.411571 19542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:59.411584 19545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:42:59.412617 19535 server_base.cc:1034] running on GCE node
I20260810 18:42:59.413364 19535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:42:59.414494 19535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:42:59.415755 19535 hybrid_clock.cc:648] HybridClock initialized: now 1786387379415687 us; error 49 us; skew 500 ppm
I20260810 18:42:59.416203 19535 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:42:59.419126 19535 webserver.cc:460] Webserver started at http://127.4.228.194:35339/ using document root <none> and password file <none>
I20260810 18:42:59.419982 19535 fs_manager.cc:359] Metadata directory not provided
I20260810 18:42:59.420125 19535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:42:59.420512 19535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:42:59.424172 19535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/data/instance:
uuid: "d19703a4e0da426bacf42c7e41062e64"
format_stamp: "Formatted at 2026-08-10 18:42:59 on dist-test-slave-7kzw"
server_key: "5696ad1d88079b44ea241d7ed36d8285"
server_key_iv: "bc4e8f79e5242b15012eacb00e5602ec"
server_key_version: "encryptionkey@0"
I20260810 18:42:59.425175 19535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance:
uuid: "d19703a4e0da426bacf42c7e41062e64"
format_stamp: "Formatted at 2026-08-10 18:42:59 on dist-test-slave-7kzw"
server_key: "5696ad1d88079b44ea241d7ed36d8285"
server_key_iv: "bc4e8f79e5242b15012eacb00e5602ec"
server_key_version: "encryptionkey@0"
I20260810 18:42:59.431734 19535 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260810 18:42:59.436162 19554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:59.437721 19535 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.004s
I20260810 18:42:59.437930 19535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/wal
uuid: "d19703a4e0da426bacf42c7e41062e64"
format_stamp: "Formatted at 2026-08-10 18:42:59 on dist-test-slave-7kzw"
server_key: "5696ad1d88079b44ea241d7ed36d8285"
server_key_iv: "bc4e8f79e5242b15012eacb00e5602ec"
server_key_version: "encryptionkey@0"
I20260810 18:42:59.438159 19535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:42:59.462312 19535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:42:59.463317 19535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:42:59.463608 19535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:42:59.464639 19535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:42:59.466383 19535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:42:59.466498 19535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:59.466668 19535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:42:59.466748 19535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:59.503576 19535 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.194:42591
I20260810 18:42:59.503646 19737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.194:42591 every 8 connection(s)
I20260810 18:42:59.505087 19535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
I20260810 18:42:59.506142 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 19535
I20260810 18:42:59.506357 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance
I20260810 18:42:59.507022 5011 external_mini_cluster.cc:1428] Setting key 7cbc8737a22db16ec00e3754f947a8af
I20260810 18:42:59.510690 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.195:0
--local_ip_for_outbound_sockets=127.4.228.195
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:35833
--builtin_ntp_servers=127.4.228.212:32791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 18:42:59.519699 19740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:59.508487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:60795 (local address 127.4.228.254:35833)
0810 18:42:59.509054 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0810 18:42:59.509060 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:42:59.510612 (+ 1552us) server_negotiation.cc:407] Connection header received
0810 18:42:59.512518 (+ 1906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:42:59.512524 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:42:59.512622 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:42:59.512745 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:42:59.513912 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:59.514737 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:59.516362 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:42:59.516789 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:42:59.516926 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:59.518397 (+ 1471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:42:59.518402 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:42:59.518406 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:42:59.518427 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:42:59.518518 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:42:59.518536 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:42:59.518638 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 18:42:59.519038 (+ 400us) server_negotiation.cc:299] Negotiation successful
0810 18:42:59.519172 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":296,"threads_started":1}
I20260810 18:42:59.521081 19741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:42:59.509282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35833 (local address 127.4.228.194:60795)
0810 18:42:59.510397 (+ 1115us) negotiation.cc:107] Waiting for socket to connect
0810 18:42:59.510443 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 18:42:59.512231 (+ 1788us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:42:59.513016 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:42:59.513024 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:42:59.513124 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:42:59.513723 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:59.513741 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:59.514911 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:59.514916 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:59.516181 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:42:59.516195 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:42:59.517040 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:42:59.517048 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:42:59.517292 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:42:59.518038 (+ 746us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:42:59.518041 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:42:59.518086 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:42:59.518179 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:42:59.518641 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:42:59.518656 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:42:59.518666 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 18:42:59.519027 (+ 361us) client_negotiation.cc:240] Negotiation successful
0810 18:42:59.519618 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":403,"threads_started":1}
I20260810 18:42:59.522856 19738 heartbeater.cc:347] Connected to a master server at 127.4.228.254:35833
I20260810 18:42:59.523226 19738 heartbeater.cc:464] Registering TS with master...
I20260810 18:42:59.524160 19738 heartbeater.cc:511] Master 127.4.228.254:35833 requested a full tablet report, sending...
I20260810 18:42:59.526057 19251 ts_manager.cc:194] Registered new tserver with Master: d19703a4e0da426bacf42c7e41062e64 (127.4.228.194:42591)
I20260810 18:42:59.527501 19251 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.228.194:60795
W20260810 18:42:59.891652 19743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:42:59.892006 19743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:42:59.892096 19743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:42:59.899718 19743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:42:59.899847 19743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:42:59.899955 19743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.195
I20260810 18:42:59.910044 19743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:32791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:35833
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.195
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:42:59.911527 19743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:42:59.913587 19743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:42:59.929447 19757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:59.929725 19754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:42:59.930052 19766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:42:59.930577 19743 server_base.cc:1034] running on GCE node
I20260810 18:42:59.931429 19743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:42:59.932714 19743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:42:59.933959 19743 hybrid_clock.cc:648] HybridClock initialized: now 1786387379933891 us; error 59 us; skew 500 ppm
I20260810 18:42:59.934327 19743 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:42:59.937192 19743 webserver.cc:460] Webserver started at http://127.4.228.195:43567/ using document root <none> and password file <none>
I20260810 18:42:59.938071 19743 fs_manager.cc:359] Metadata directory not provided
I20260810 18:42:59.938225 19743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:42:59.938710 19743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:42:59.942075 19743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/data/instance:
uuid: "5f540fcec08c4e1d82d54bbd278e2123"
format_stamp: "Formatted at 2026-08-10 18:42:59 on dist-test-slave-7kzw"
server_key: "45e713a88f95e788ad446d7a15b79a91"
server_key_iv: "b97553fcc6010959564fe000e601ee2f"
server_key_version: "encryptionkey@0"
I20260810 18:42:59.943100 19743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance:
uuid: "5f540fcec08c4e1d82d54bbd278e2123"
format_stamp: "Formatted at 2026-08-10 18:42:59 on dist-test-slave-7kzw"
server_key: "45e713a88f95e788ad446d7a15b79a91"
server_key_iv: "b97553fcc6010959564fe000e601ee2f"
server_key_version: "encryptionkey@0"
I20260810 18:42:59.949345 19743 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 18:42:59.953342 19775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:59.955160 19743 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:42:59.955384 19743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/wal
uuid: "5f540fcec08c4e1d82d54bbd278e2123"
format_stamp: "Formatted at 2026-08-10 18:42:59 on dist-test-slave-7kzw"
server_key: "45e713a88f95e788ad446d7a15b79a91"
server_key_iv: "b97553fcc6010959564fe000e601ee2f"
server_key_version: "encryptionkey@0"
I20260810 18:42:59.955641 19743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:42:59.968161 19743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:42:59.969380 19743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:42:59.969762 19743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:42:59.971024 19743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:42:59.972764 19743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:42:59.972867 19743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:42:59.973021 19743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:42:59.973124 19743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:43:00.013425 19528 heartbeater.cc:503] Master 127.4.228.254:35833 was elected leader, sending a full tablet report...
I20260810 18:43:00.014503 19743 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.195:40337
I20260810 18:43:00.014617 19947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.195:40337 every 8 connection(s)
I20260810 18:43:00.016191 19743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
I20260810 18:43:00.026177 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 19743
I20260810 18:43:00.026397 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance
I20260810 18:43:00.026899 5011 external_mini_cluster.cc:1428] Setting key 6fcd3982a5bfcda2876e47503f9db0bb
I20260810 18:43:00.032773 19740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:43:00.019538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:59073 (local address 127.4.228.254:35833)
0810 18:43:00.019768 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0810 18:43:00.019775 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:43:00.021501 (+ 1726us) server_negotiation.cc:407] Connection header received
0810 18:43:00.023954 (+ 2453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:43:00.023960 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:43:00.024045 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:43:00.024163 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:43:00.026094 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:43:00.027131 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:43:00.029240 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:43:00.029848 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:43:00.030002 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:43:00.031893 (+ 1891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:43:00.031898 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:43:00.031902 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:43:00.031941 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:43:00.031977 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:43:00.031991 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:43:00.032066 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 18:43:00.032411 (+ 345us) server_negotiation.cc:299] Negotiation successful
0810 18:43:00.032505 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 18:43:00.034401 19955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:43:00.020088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35833 (local address 127.4.228.195:59073)
0810 18:43:00.021307 (+ 1219us) negotiation.cc:107] Waiting for socket to connect
0810 18:43:00.021359 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 18:43:00.023451 (+ 2092us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:43:00.024569 (+ 1118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:43:00.024582 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:43:00.024729 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:43:00.025818 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:43:00.025853 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:43:00.027355 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:43:00.027363 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:43:00.029039 (+ 1676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:43:00.029061 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:43:00.030134 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:43:00.030145 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:43:00.030460 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:43:00.031538 (+ 1078us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:43:00.031541 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:43:00.031595 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:43:00.031703 (+ 108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:43:00.032113 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:43:00.032123 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:43:00.032132 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 18:43:00.032373 (+ 241us) client_negotiation.cc:240] Negotiation successful
0810 18:43:00.032770 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":710,"thread_start_us":390,"threads_started":1}
I20260810 18:43:00.036361 19950 heartbeater.cc:347] Connected to a master server at 127.4.228.254:35833
I20260810 18:43:00.036836 19950 heartbeater.cc:464] Registering TS with master...
I20260810 18:43:00.037776 19950 heartbeater.cc:511] Master 127.4.228.254:35833 requested a full tablet report, sending...
I20260810 18:43:00.039988 19251 ts_manager.cc:194] Registered new tserver with Master: 5f540fcec08c4e1d82d54bbd278e2123 (127.4.228.195:40337)
I20260810 18:43:00.040830 19251 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.228.195:59073
I20260810 18:43:00.044420 5011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 18:43:00.052269 19740 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:43:00.050001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44892 (local address 127.4.228.254:35833)
0810 18:43:00.050345 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0810 18:43:00.050356 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:43:00.050922 (+ 566us) server_negotiation.cc:407] Connection header received
0810 18:43:00.051105 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:43:00.051109 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:43:00.051185 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:43:00.051298 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:43:00.051988 (+ 690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44892: BlockingRecv error: recv got EOF from 127.0.0.1:44892 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
I20260810 18:43:00.052670 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 19344
I20260810 18:43:00.062004 19962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:43:00.053943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42048 (local address 127.4.228.193:40517)
0810 18:43:00.054589 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0810 18:43:00.054596 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:43:00.054625 (+ 29us) server_negotiation.cc:407] Connection header received
0810 18:43:00.054686 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:43:00.054695 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:43:00.054928 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:43:00.055057 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:43:00.055525 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:43:00.056571 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:43:00.057941 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:43:00.058676 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:43:00.058900 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:43:00.060285 (+ 1385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:43:00.060295 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:43:00.060315 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:43:00.061126 (+ 811us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:43:00.061181 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:43:00.061204 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:43:00.061309 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 18:43:00.061508 (+ 199us) server_negotiation.cc:299] Negotiation successful
0810 18:43:00.061635 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":307,"threads_started":1}
I20260810 18:43:00.063462 19519 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:43:00.157212 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 19344
I20260810 18:43:00.185107 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 19535
I20260810 18:43:00.195046 19964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:43:00.186196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34170 (local address 127.4.228.194:42591)
0810 18:43:00.186856 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0810 18:43:00.186864 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:43:00.186894 (+ 30us) server_negotiation.cc:407] Connection header received
0810 18:43:00.186957 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:43:00.186963 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:43:00.187202 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:43:00.187365 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:43:00.187897 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:43:00.188966 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:43:00.190435 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:43:00.191057 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:43:00.191182 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:43:00.192936 (+ 1754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:43:00.192947 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:43:00.192965 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:43:00.193769 (+ 804us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:43:00.193856 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:43:00.193880 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:43:00.193992 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 18:43:00.194556 (+ 564us) server_negotiation.cc:299] Negotiation successful
0810 18:43:00.194708 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":385,"threads_started":1}
I20260810 18:43:00.196874 19730 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:43:00.287657 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 19535
I20260810 18:43:00.321162 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 19743
I20260810 18:43:00.331359 19969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:43:00.322432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47222 (local address 127.4.228.195:40337)
0810 18:43:00.323324 (+ 892us) server_negotiation.cc:206] Beginning negotiation
0810 18:43:00.323332 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:43:00.323366 (+ 34us) server_negotiation.cc:407] Connection header received
0810 18:43:00.323433 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:43:00.323439 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:43:00.323741 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:43:00.323900 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:43:00.324520 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:43:00.325686 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:43:00.327353 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:43:00.328027 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:43:00.328258 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:43:00.329568 (+ 1310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:43:00.329578 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:43:00.329595 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:43:00.330269 (+ 674us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:43:00.330325 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:43:00.330348 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:43:00.330456 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 18:43:00.330757 (+ 301us) server_negotiation.cc:299] Negotiation successful
0810 18:43:00.330893 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":728,"thread_start_us":426,"threads_started":1}
I20260810 18:43:00.332912 19942 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:43:00.420027 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 19743
I20260810 18:43:00.447619 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 19197
I20260810 18:43:00.449070 19299 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:43:00.560078 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 19197
2026-08-10T18:43:00Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (3982 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-10T18:43:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:43:00Z Disabled control of system clock
I20260810 18:43:00.669080 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:36981
--webserver_interface=127.4.228.254
--webserver_port=0
--builtin_ntp_servers=127.4.228.212:45435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:36981
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 18:43:01.057287 19978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:43:01.057626 19978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:43:01.057760 19978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:43:01.065435 19978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:43:01.065572 19978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:43:01.065615 19978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:43:01.065735 19978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:43:01.065799 19978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:43:01.075774 19978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:45435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:36981
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:36981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:43:01.077265 19978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:43:01.079334 19978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:43:01.092689 19987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:43:01.092710 19988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:43:01.092682 19990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:43:01.093631 19978 server_base.cc:1034] running on GCE node
I20260810 18:43:01.094528 19978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:43:01.096127 19978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:43:01.097368 19978 hybrid_clock.cc:648] HybridClock initialized: now 1786387381097322 us; error 35 us; skew 500 ppm
I20260810 18:43:01.097780 19978 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:43:01.100411 19978 webserver.cc:460] Webserver started at http://127.4.228.254:45865/ using document root <none> and password file <none>
I20260810 18:43:01.101151 19978 fs_manager.cc:359] Metadata directory not provided
I20260810 18:43:01.101244 19978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:43:01.101538 19978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:43:01.104723 19978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "cf14b28d447d4ae891378e6b0594b4c4"
format_stamp: "Formatted at 2026-08-10 18:43:01 on dist-test-slave-7kzw"
server_key: "d613eef12a2c9dd8d49c01b11bb364f8"
server_key_iv: "7789ed673b441b48c21cc1bf16384c48"
server_key_version: "encryptionkey@0"
I20260810 18:43:01.105540 19978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "cf14b28d447d4ae891378e6b0594b4c4"
format_stamp: "Formatted at 2026-08-10 18:43:01 on dist-test-slave-7kzw"
server_key: "d613eef12a2c9dd8d49c01b11bb364f8"
server_key_iv: "7789ed673b441b48c21cc1bf16384c48"
server_key_version: "encryptionkey@0"
I20260810 18:43:01.111804 19978 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260810 18:43:01.115763 20000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:43:01.117411 19978 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:43:01.117579 19978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "cf14b28d447d4ae891378e6b0594b4c4"
format_stamp: "Formatted at 2026-08-10 18:43:01 on dist-test-slave-7kzw"
server_key: "d613eef12a2c9dd8d49c01b11bb364f8"
server_key_iv: "7789ed673b441b48c21cc1bf16384c48"
server_key_version: "encryptionkey@0"
I20260810 18:43:01.117803 19978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:43:01.124332 19978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:43:01.125221 19978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:43:01.125547 19978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:43:01.147017 19978 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:36981
I20260810 18:43:01.147142 20080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:36981 every 8 connection(s)
I20260810 18:43:01.148705 19978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:43:01.152886 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 19978
I20260810 18:43:01.152936 20082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:43:01.153110 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:43:01.153870 5011 external_mini_cluster.cc:1428] Setting key fc39c4db0006b7f2feb62b9b31994ed2
I20260810 18:43:01.164705 20085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:43:01.156299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60662 (local address 127.4.228.254:36981)
0810 18:43:01.157543 (+ 1244us) server_negotiation.cc:206] Beginning negotiation
0810 18:43:01.157555 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:43:01.157623 (+ 68us) server_negotiation.cc:407] Connection header received
0810 18:43:01.159992 (+ 2369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:43:01.160024 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:43:01.160654 (+ 630us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:43:01.161171 (+ 517us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:43:01.162572 (+ 1401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:43:01.162587 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:43:01.162602 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:43:01.162619 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:43:01.162689 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:43:01.162707 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:43:01.163092 (+ 385us) server_negotiation.cc:1035] Waiting for connection context
0810 18:43:01.163264 (+ 172us) server_negotiation.cc:299] Negotiation successful
0810 18:43:01.163563 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":708,"thread_start_us":433,"threads_started":1}
I20260810 18:43:01.168061 20082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf14b28d447d4ae891378e6b0594b4c4: Bootstrap starting.
I20260810 18:43:01.171651 20082 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf14b28d447d4ae891378e6b0594b4c4: Neither blocks nor log segments found. Creating new log.
I20260810 18:43:01.172724 20082 log.cc:824] T 00000000000000000000000000000000 P cf14b28d447d4ae891378e6b0594b4c4: Log is configured to *not* fsync() on all Append() calls
I20260810 18:43:01.176180 20082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf14b28d447d4ae891378e6b0594b4c4: No bootstrap required, opened a new log
I20260810 18:43:01.184208 20082 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf14b28d447d4ae891378e6b0594b4c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf14b28d447d4ae891378e6b0594b4c4" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 36981 } }
I20260810 18:43:01.184614 20082 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf14b28d447d4ae891378e6b0594b4c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:43:01.184693 20082 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf14b28d447d4ae891378e6b0594b4c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf14b28d447d4ae891378e6b0594b4c4, State: Initialized, Role: FOLLOWER
I20260810 18:43:01.185266 20082 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf14b28d447d4ae891378e6b0594b4c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf14b28d447d4ae891378e6b0594b4c4" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 36981 } }
I20260810 18:43:01.185482 20082 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf14b28d447d4ae891378e6b0594b4c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:43:01.185587 20082 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf14b28d447d4ae891378e6b0594b4c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:43:01.185726 20082 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf14b28d447d4ae891378e6b0594b4c4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:43:01.188264 20082 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf14b28d447d4ae891378e6b0594b4c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf14b28d447d4ae891378e6b0594b4c4" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 36981 } }
I20260810 18:43:01.188796 20082 leader_election.cc:302] T 00000000000000000000000000000000 P cf14b28d447d4ae891378e6b0594b4c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf14b28d447d4ae891378e6b0594b4c4; no voters:
I20260810 18:43:01.189487 20082 leader_election.cc:288] T 00000000000000000000000000000000 P cf14b28d447d4ae891378e6b0594b4c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:43:01.189707 20091 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf14b28d447d4ae891378e6b0594b4c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:43:01.190516 20091 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf14b28d447d4ae891378e6b0594b4c4 [term 1 LEADER]: Becoming Leader. State: Replica: cf14b28d447d4ae891378e6b0594b4c4, State: Running, Role: LEADER
I20260810 18:43:01.191116 20091 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf14b28d447d4ae891378e6b0594b4c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf14b28d447d4ae891378e6b0594b4c4" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 36981 } }
I20260810 18:43:01.191891 20082 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf14b28d447d4ae891378e6b0594b4c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:43:01.194203 20094 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf14b28d447d4ae891378e6b0594b4c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf14b28d447d4ae891378e6b0594b4c4. Latest consensus state: current_term: 1 leader_uuid: "cf14b28d447d4ae891378e6b0594b4c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf14b28d447d4ae891378e6b0594b4c4" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 36981 } } }
I20260810 18:43:01.194645 20094 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf14b28d447d4ae891378e6b0594b4c4 [sys.catalog]: This master's current role is: LEADER
I20260810 18:43:01.196383 20092 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf14b28d447d4ae891378e6b0594b4c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf14b28d447d4ae891378e6b0594b4c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf14b28d447d4ae891378e6b0594b4c4" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 36981 } } }
I20260810 18:43:01.197445 20092 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf14b28d447d4ae891378e6b0594b4c4 [sys.catalog]: This master's current role is: LEADER
I20260810 18:43:01.197595 20104 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:43:01.204638 20104 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:43:01.215445 20104 catalog_manager.cc:1347] Generated new cluster ID: a4e94f00d0a54be8817f15787c966830
I20260810 18:43:01.215622 20104 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:43:01.236768 20104 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:43:01.237352 20104 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:43:01.256774 20104 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf14b28d447d4ae891378e6b0594b4c4: Generated new TSK 0
I20260810 18:43:01.257844 20104 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:43:01.271802 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.193:0
--local_ip_for_outbound_sockets=127.4.228.193
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:36981
--builtin_ntp_servers=127.4.228.212:45435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 18:43:01.652573 20126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:43:01.653020 20126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:43:01.653127 20126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:43:01.660611 20126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:43:01.660745 20126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:43:01.660868 20126 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.193
I20260810 18:43:01.674445 20126 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:45435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:36981
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.193
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:43:01.675995 20126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:43:01.678033 20126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:43:01.693060 20133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:43:01.693161 20132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:43:01.693112 20138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:43:01.694375 20126 server_base.cc:1034] running on GCE node
I20260810 18:43:01.695207 20126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:43:01.696386 20126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:43:01.697657 20126 hybrid_clock.cc:648] HybridClock initialized: now 1786387381697578 us; error 69 us; skew 500 ppm
I20260810 18:43:01.698035 20126 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:43:01.706131 20126 webserver.cc:460] Webserver started at http://127.4.228.193:44123/ using document root <none> and password file <none>
I20260810 18:43:01.707069 20126 fs_manager.cc:359] Metadata directory not provided
I20260810 18:43:01.707237 20126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:43:01.707638 20126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:43:01.711002 20126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/data/instance:
uuid: "0451aef6c1fa45029e63c214424cc60a"
format_stamp: "Formatted at 2026-08-10 18:43:01 on dist-test-slave-7kzw"
server_key: "ba64e5a165d2a9e8c47d9071186bd3ae"
server_key_iv: "22448a3cf8e638a5d691a725b701a196"
server_key_version: "encryptionkey@0"
I20260810 18:43:01.711942 20126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance:
uuid: "0451aef6c1fa45029e63c214424cc60a"
format_stamp: "Formatted at 2026-08-10 18:43:01 on dist-test-slave-7kzw"
server_key: "ba64e5a165d2a9e8c47d9071186bd3ae"
server_key_iv: "22448a3cf8e638a5d691a725b701a196"
server_key_version: "encryptionkey@0"
I20260810 18:43:01.717761 20126 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260810 18:43:01.721881 20147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:43:01.723409 20126 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:43:01.723605 20126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/wal
uuid: "0451aef6c1fa45029e63c214424cc60a"
format_stamp: "Formatted at 2026-08-10 18:43:01 on dist-test-slave-7kzw"
server_key: "ba64e5a165d2a9e8c47d9071186bd3ae"
server_key_iv: "22448a3cf8e638a5d691a725b701a196"
server_key_version: "encryptionkey@0"
I20260810 18:43:01.723858 20126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:43:01.730811 20126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:43:01.731730 20126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:43:01.732031 20126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:43:01.733067 20126 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:43:01.734774 20126 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:43:01.734898 20126 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:43:01.735049 20126 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:43:01.735141 20126 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:43:01.772217 20126 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.193:39333
I20260810 18:43:01.772320 20326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.193:39333 every 8 connection(s)
I20260810 18:43:01.773693 20126 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
I20260810 18:43:01.775625 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 20126
I20260810 18:43:01.775844 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance
I20260810 18:43:01.776675 5011 external_mini_cluster.cc:1428] Setting key 904ecf8b4ff883c2ee57ba5b3241f984
I20260810 18:43:01.780119 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.194:0
--local_ip_for_outbound_sockets=127.4.228.194
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:36981
--builtin_ntp_servers=127.4.228.212:45435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260810 18:43:01.783700 20329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:43:01.777093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:42981 (local address 127.4.228.254:36981)
0810 18:43:01.777709 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0810 18:43:01.777717 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:43:01.779237 (+ 1520us) server_negotiation.cc:407] Connection header received
0810 18:43:01.781176 (+ 1939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:43:01.781182 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:43:01.781253 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:43:01.781393 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:43:01.782871 (+ 1478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:43:01.782877 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:43:01.782882 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:43:01.782903 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:43:01.782958 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:43:01.782970 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:43:01.783112 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 18:43:01.783316 (+ 204us) server_negotiation.cc:299] Negotiation successful
0810 18:43:01.783394 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":358,"threads_started":1}
I20260810 18:43:01.784654 20330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:43:01.777800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:36981 (local address 127.4.228.193:42981)
0810 18:43:01.778993 (+ 1193us) negotiation.cc:107] Waiting for socket to connect
0810 18:43:01.779042 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 18:43:01.780894 (+ 1852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:43:01.781651 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:43:01.781660 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:43:01.781764 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:43:01.782554 (+ 790us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:43:01.782557 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:43:01.782599 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:43:01.782697 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:43:01.783073 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:43:01.783086 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:43:01.783096 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 18:43:01.783405 (+ 309us) client_negotiation.cc:240] Negotiation successful
0810 18:43:01.783672 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":377,"threads_started":1}
I20260810 18:43:01.786356 20327 heartbeater.cc:347] Connected to a master server at 127.4.228.254:36981
I20260810 18:43:01.786757 20327 heartbeater.cc:464] Registering TS with master...
I20260810 18:43:01.787611 20327 heartbeater.cc:511] Master 127.4.228.254:36981 requested a full tablet report, sending...
I20260810 18:43:01.790365 20024 ts_manager.cc:194] Registered new tserver with Master: 0451aef6c1fa45029e63c214424cc60a (127.4.228.193:39333)
W20260810 18:43:02.161623 20333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:43:02.161918 20333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:43:02.162075 20333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:43:02.169476 20333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:43:02.169582 20333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:43:02.169759 20333 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.194
I20260810 18:43:02.179513 20333 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:45435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:36981
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.194
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:43:02.180982 20333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:43:02.183027 20333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:43:02.197525 20342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:43:02.197553 20338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:43:02.197631 20344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:43:02.198860 20333 server_base.cc:1034] running on GCE node
I20260810 18:43:02.199852 20333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:43:02.201056 20333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:43:02.202340 20333 hybrid_clock.cc:648] HybridClock initialized: now 1786387382202282 us; error 56 us; skew 500 ppm
I20260810 18:43:02.202800 20333 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:43:02.205737 20333 webserver.cc:460] Webserver started at http://127.4.228.194:41489/ using document root <none> and password file <none>
I20260810 18:43:02.206704 20333 fs_manager.cc:359] Metadata directory not provided
I20260810 18:43:02.206892 20333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:43:02.207334 20333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:43:02.210760 20333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/data/instance:
uuid: "3e470e37cec246e99648140c161b782f"
format_stamp: "Formatted at 2026-08-10 18:43:02 on dist-test-slave-7kzw"
server_key: "534e7a4175c3563241479d0bc7ab7ece"
server_key_iv: "b6f30502987d52403da11a6fde60d9e6"
server_key_version: "encryptionkey@0"
I20260810 18:43:02.211704 20333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance:
uuid: "3e470e37cec246e99648140c161b782f"
format_stamp: "Formatted at 2026-08-10 18:43:02 on dist-test-slave-7kzw"
server_key: "534e7a4175c3563241479d0bc7ab7ece"
server_key_iv: "b6f30502987d52403da11a6fde60d9e6"
server_key_version: "encryptionkey@0"
I20260810 18:43:02.217770 20333 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260810 18:43:02.221541 20351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:43:02.223227 20333 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:43:02.223366 20333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/wal
uuid: "3e470e37cec246e99648140c161b782f"
format_stamp: "Formatted at 2026-08-10 18:43:02 on dist-test-slave-7kzw"
server_key: "534e7a4175c3563241479d0bc7ab7ece"
server_key_iv: "b6f30502987d52403da11a6fde60d9e6"
server_key_version: "encryptionkey@0"
I20260810 18:43:02.223557 20333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:43:02.230085 20333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:43:02.230999 20333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:43:02.231287 20333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:43:02.232242 20333 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:43:02.233889 20333 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:43:02.233983 20333 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:43:02.234063 20333 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:43:02.234119 20333 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:43:02.271435 20333 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.194:46039
I20260810 18:43:02.271481 20521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.194:46039 every 8 connection(s)
I20260810 18:43:02.272861 20333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
I20260810 18:43:02.273873 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 20333
I20260810 18:43:02.274048 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance
I20260810 18:43:02.274684 5011 external_mini_cluster.cc:1428] Setting key 7964506b5fe97c186b6db721ed8154e4
I20260810 18:43:02.278033 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.195:0
--local_ip_for_outbound_sockets=127.4.228.195
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:36981
--builtin_ntp_servers=127.4.228.212:45435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260810 18:43:02.282757 20329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:43:02.276047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:55599 (local address 127.4.228.254:36981)
0810 18:43:02.276272 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0810 18:43:02.276280 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:43:02.278058 (+ 1778us) server_negotiation.cc:407] Connection header received
0810 18:43:02.279955 (+ 1897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:43:02.279964 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:43:02.280059 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:43:02.280190 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:43:02.281737 (+ 1547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:43:02.281747 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:43:02.281755 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:43:02.281785 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:43:02.281834 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:43:02.281856 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:43:02.282017 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 18:43:02.282266 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 18:43:02.282350 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 18:43:02.283888 20524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:43:02.276677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:36981 (local address 127.4.228.194:55599)
0810 18:43:02.277843 (+ 1166us) negotiation.cc:107] Waiting for socket to connect
0810 18:43:02.277901 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 18:43:02.279638 (+ 1737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:43:02.280465 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:43:02.280475 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:43:02.280591 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:43:02.281372 (+ 781us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:43:02.281375 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:43:02.281418 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:43:02.281515 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:43:02.281995 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:43:02.282005 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:43:02.282011 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 18:43:02.282302 (+ 291us) client_negotiation.cc:240] Negotiation successful
0810 18:43:02.282819 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":391,"threads_started":1}
I20260810 18:43:02.285759 20522 heartbeater.cc:347] Connected to a master server at 127.4.228.254:36981
I20260810 18:43:02.286161 20522 heartbeater.cc:464] Registering TS with master...
I20260810 18:43:02.287010 20522 heartbeater.cc:511] Master 127.4.228.254:36981 requested a full tablet report, sending...
I20260810 18:43:02.289036 20024 ts_manager.cc:194] Registered new tserver with Master: 3e470e37cec246e99648140c161b782f (127.4.228.194:46039)
W20260810 18:43:02.637494 20526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:43:02.637859 20526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:43:02.638041 20526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:43:02.645421 20526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:43:02.645581 20526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:43:02.645783 20526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.195
I20260810 18:43:02.655256 20526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:45435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:36981
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.195
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:43:02.656754 20526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:43:02.658771 20526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:43:02.676687 20533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:43:02.677286 20534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:43:02.677532 20537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:43:02.678669 20526 server_base.cc:1034] running on GCE node
I20260810 18:43:02.679539 20526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:43:02.681015 20526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:43:02.682314 20526 hybrid_clock.cc:648] HybridClock initialized: now 1786387382682248 us; error 60 us; skew 500 ppm
I20260810 18:43:02.682796 20526 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:43:02.685802 20526 webserver.cc:460] Webserver started at http://127.4.228.195:37573/ using document root <none> and password file <none>
I20260810 18:43:02.686975 20526 fs_manager.cc:359] Metadata directory not provided
I20260810 18:43:02.687120 20526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:43:02.687628 20526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:43:02.691948 20526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/data/instance:
uuid: "019c40ae5f3e4360bd146f93547f8a9c"
format_stamp: "Formatted at 2026-08-10 18:43:02 on dist-test-slave-7kzw"
server_key: "09a8d941e6ede5847c3a2ec4b598c715"
server_key_iv: "94b09710c1b5ed86bbeeef7282b177bf"
server_key_version: "encryptionkey@0"
I20260810 18:43:02.693408 20526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance:
uuid: "019c40ae5f3e4360bd146f93547f8a9c"
format_stamp: "Formatted at 2026-08-10 18:43:02 on dist-test-slave-7kzw"
server_key: "09a8d941e6ede5847c3a2ec4b598c715"
server_key_iv: "94b09710c1b5ed86bbeeef7282b177bf"
server_key_version: "encryptionkey@0"
I20260810 18:43:02.700222 20526 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.003s
I20260810 18:43:02.704739 20546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:43:02.706347 20526 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:43:02.706598 20526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/wal
uuid: "019c40ae5f3e4360bd146f93547f8a9c"
format_stamp: "Formatted at 2026-08-10 18:43:02 on dist-test-slave-7kzw"
server_key: "09a8d941e6ede5847c3a2ec4b598c715"
server_key_iv: "94b09710c1b5ed86bbeeef7282b177bf"
server_key_version: "encryptionkey@0"
I20260810 18:43:02.706887 20526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:43:02.713182 20526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:43:02.714206 20526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:43:02.714629 20526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:43:02.715734 20526 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:43:02.717379 20526 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:43:02.717514 20526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:43:02.717662 20526 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:43:02.717759 20526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:43:02.754233 20526 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.195:37751
I20260810 18:43:02.754333 20731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.195:37751 every 8 connection(s)
I20260810 18:43:02.755707 20526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
I20260810 18:43:02.762743 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 20526
I20260810 18:43:02.762941 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance
I20260810 18:43:02.763569 5011 external_mini_cluster.cc:1428] Setting key 2382f36bccc7cfae561004ee9fb2ed3f
I20260810 18:43:02.765414 20329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:43:02.758601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:60753 (local address 127.4.228.254:36981)
0810 18:43:02.758789 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 18:43:02.758795 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:43:02.760344 (+ 1549us) server_negotiation.cc:407] Connection header received
0810 18:43:02.762341 (+ 1997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:43:02.762349 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:43:02.762434 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:43:02.762594 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:43:02.764385 (+ 1791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:43:02.764392 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:43:02.764397 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:43:02.764423 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:43:02.764472 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:43:02.764494 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:43:02.764624 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 18:43:02.764946 (+ 322us) server_negotiation.cc:299] Negotiation successful
0810 18:43:02.765019 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 18:43:02.766649 20735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:43:02.759082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:36981 (local address 127.4.228.195:60753)
0810 18:43:02.760137 (+ 1055us) negotiation.cc:107] Waiting for socket to connect
0810 18:43:02.760189 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 18:43:02.761975 (+ 1786us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:43:02.762906 (+ 931us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:43:02.762919 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:43:02.763054 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:43:02.763956 (+ 902us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:43:02.763961 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:43:02.764017 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:43:02.764146 (+ 129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:43:02.764655 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:43:02.764667 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:43:02.764676 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 18:43:02.765017 (+ 341us) client_negotiation.cc:240] Negotiation successful
0810 18:43:02.765361 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":322,"threads_started":1}
I20260810 18:43:02.768232 20733 heartbeater.cc:347] Connected to a master server at 127.4.228.254:36981
I20260810 18:43:02.768532 20733 heartbeater.cc:464] Registering TS with master...
I20260810 18:43:02.769147 20733 heartbeater.cc:511] Master 127.4.228.254:36981 requested a full tablet report, sending...
I20260810 18:43:02.770812 20024 ts_manager.cc:194] Registered new tserver with Master: 019c40ae5f3e4360bd146f93547f8a9c (127.4.228.195:37751)
I20260810 18:43:02.780633 5011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 18:43:02.787608 20329 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:43:02.785825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60670 (local address 127.4.228.254:36981)
0810 18:43:02.786015 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 18:43:02.786020 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:43:02.786444 (+ 424us) server_negotiation.cc:407] Connection header received
0810 18:43:02.786645 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:43:02.786650 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:43:02.786723 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:43:02.786826 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:43:02.787311 (+ 485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60670: BlockingRecv error: recv got EOF from 127.0.0.1:60670 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 18:43:02.787969 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 20126
I20260810 18:43:02.792689 20743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:43:02.789605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52264 (local address 127.4.228.193:39333)
0810 18:43:02.790227 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0810 18:43:02.790234 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:43:02.790269 (+ 35us) server_negotiation.cc:407] Connection header received
0810 18:43:02.790332 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:43:02.790339 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:43:02.790656 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:43:02.790792 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:43:02.791798 (+ 1006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:43:02.791806 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:43:02.791817 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:43:02.791829 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:43:02.791874 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:43:02.791888 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:43:02.792088 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0810 18:43:02.792280 (+ 192us) server_negotiation.cc:299] Negotiation successful
0810 18:43:02.792404 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":303,"threads_started":1}
I20260810 18:43:02.793989 20327 heartbeater.cc:503] Master 127.4.228.254:36981 was elected leader, sending a full tablet report...
I20260810 18:43:02.794102 20320 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:43:02.878800 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 20126
I20260810 18:43:02.907027 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 20333
I20260810 18:43:02.911550 20745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:43:02.908244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44102 (local address 127.4.228.194:46039)
0810 18:43:02.908956 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0810 18:43:02.908964 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:43:02.908996 (+ 32us) server_negotiation.cc:407] Connection header received
0810 18:43:02.909067 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:43:02.909074 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:43:02.909349 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:43:02.909500 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:43:02.910742 (+ 1242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:43:02.910751 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:43:02.910765 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:43:02.910791 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:43:02.910845 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:43:02.910863 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:43:02.911026 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 18:43:02.911213 (+ 187us) server_negotiation.cc:299] Negotiation successful
0810 18:43:02.911305 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":398,"threads_started":1}
I20260810 18:43:02.913218 20513 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:43:02.999488 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 20333
I20260810 18:43:03.026063 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 20526
I20260810 18:43:03.030826 20747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:43:03.027199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55746 (local address 127.4.228.195:37751)
0810 18:43:03.027894 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0810 18:43:03.027902 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:43:03.027935 (+ 33us) server_negotiation.cc:407] Connection header received
0810 18:43:03.028003 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:43:03.028010 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:43:03.028248 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:43:03.028378 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:43:03.029777 (+ 1399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:43:03.029784 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:43:03.029797 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:43:03.029810 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:43:03.029853 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:43:03.029872 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:43:03.030018 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 18:43:03.030223 (+ 205us) server_negotiation.cc:299] Negotiation successful
0810 18:43:03.030400 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":345,"threads_started":1}
I20260810 18:43:03.032266 20722 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:43:03.114995 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 20526
I20260810 18:43:03.141459 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 19978
I20260810 18:43:03.143090 20075 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:43:03.248862 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 19978
2026-08-10T18:43:03Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2683 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:43:03 dist-test-slave-7kzw krb5kdc[20753](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:43:03 dist-test-slave-7kzw krb5kdc[20753](info): set up 2 sockets
Aug 10 18:43:03 dist-test-slave-7kzw krb5kdc[20753](info): commencing operation
krb5kdc: starting...
W20260810 18:43:04.631757 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.331s user 0.002s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:43:04 dist-test-slave-7kzw krb5kdc[20753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:43:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:43:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.228.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
I20260810 18:43:04.979833 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:39209
--webserver_interface=127.4.228.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:44041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:39209
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:43:05.354703 20780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:43:05.355057 20780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:43:05.355230 20780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:43:05.362912 20780 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:43:05.363040 20780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:43:05.363133 20780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:43:05.363220 20780 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:43:05.363286 20780 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:43:05.374137 20780 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:44041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:39209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:39209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:43:05.375732 20780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:43:05.377257 20780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:43:05.390618 20787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:43:05.390782 20795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:43:05.390807 20791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:43:05.391922 20780 server_base.cc:1034] running on GCE node
I20260810 18:43:05.393085 20780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:43:05.394894 20780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:43:05.396196 20780 hybrid_clock.cc:648] HybridClock initialized: now 1786387385396130 us; error 49 us; skew 500 ppm
I20260810 18:43:05.396672 20780 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:43:05 dist-test-slave-7kzw krb5kdc[20753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387385, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:43:05.400575 20780 init.cc:377] Logged in from keytab as kudu/127.4.228.254@KRBTEST.COM (short username kudu)
I20260810 18:43:05.402416 20780 webserver.cc:460] Webserver started at http://127.4.228.254:34661/ using document root <none> and password file <none>
I20260810 18:43:05.403263 20780 fs_manager.cc:359] Metadata directory not provided
I20260810 18:43:05.403411 20780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:43:05.403810 20780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:43:05.407336 20780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "6a36c9ad578b45af9c80bc9e754b7105"
format_stamp: "Formatted at 2026-08-10 18:43:05 on dist-test-slave-7kzw"
server_key: "a7e0129c39150b4a4dba9fe162b89324"
server_key_iv: "8fb5159d50a686b8003194bd4c39281a"
server_key_version: "encryptionkey@0"
I20260810 18:43:05.408227 20780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "6a36c9ad578b45af9c80bc9e754b7105"
format_stamp: "Formatted at 2026-08-10 18:43:05 on dist-test-slave-7kzw"
server_key: "a7e0129c39150b4a4dba9fe162b89324"
server_key_iv: "8fb5159d50a686b8003194bd4c39281a"
server_key_version: "encryptionkey@0"
I20260810 18:43:05.414932 20780 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260810 18:43:05.419024 20808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:43:05.421624 20780 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260810 18:43:05.421861 20780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "6a36c9ad578b45af9c80bc9e754b7105"
format_stamp: "Formatted at 2026-08-10 18:43:05 on dist-test-slave-7kzw"
server_key: "a7e0129c39150b4a4dba9fe162b89324"
server_key_iv: "8fb5159d50a686b8003194bd4c39281a"
server_key_version: "encryptionkey@0"
I20260810 18:43:05.422156 20780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:43:05.441051 20780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:43:05.446660 20780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:43:05.447045 20780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:43:05.469477 20780 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:39209
I20260810 18:43:05.469499 20888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:39209 every 8 connection(s)
I20260810 18:43:05.471221 20780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:43:05.473717 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 20780
I20260810 18:43:05.473992 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:43:05.474824 5011 external_mini_cluster.cc:1428] Setting key 8dca38b6133f21606790b5cb4892b90e
I20260810 18:43:05.476027 20889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:43:05.489790 20889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a36c9ad578b45af9c80bc9e754b7105: Bootstrap starting.
Aug 10 18:43:05 dist-test-slave-7kzw krb5kdc[20753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:43:05.493772 20889 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a36c9ad578b45af9c80bc9e754b7105: Neither blocks nor log segments found. Creating new log.
I20260810 18:43:05.495131 20889 log.cc:824] T 00000000000000000000000000000000 P 6a36c9ad578b45af9c80bc9e754b7105: Log is configured to *not* fsync() on all Append() calls
I20260810 18:43:05.498390 20889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a36c9ad578b45af9c80bc9e754b7105: No bootstrap required, opened a new log
I20260810 18:43:05.501086 20894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:43:05.477121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40242 (local address 127.4.228.254:39209)
0810 18:43:05.478049 (+ 928us) server_negotiation.cc:206] Beginning negotiation
0810 18:43:05.478063 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:43:05.478121 (+ 58us) server_negotiation.cc:407] Connection header received
0810 18:43:05.480088 (+ 1967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:43:05.480125 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:43:05.480624 (+ 499us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:43:05.481147 (+ 523us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:43:05.483049 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:43:05.484805 (+ 1756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:43:05.486273 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:43:05.487117 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:43:05.487385 (+ 268us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:43:05.491666 (+ 4281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:43:05.491675 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:43:05.491691 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:43:05.491743 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:43:05.495502 (+ 3759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:43:05.496476 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:43:05.496485 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:43:05.496493 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:43:05.496713 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:43:05.497245 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:43:05.497251 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:43:05.497262 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:43:05.497926 (+ 664us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:43:05.498336 (+ 410us) server_negotiation.cc:1035] Waiting for connection context
0810 18:43:05.499298 (+ 962us) server_negotiation.cc:299] Negotiation successful
0810 18:43:05.499960 (+ 662us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":424,"threads_started":1}
I20260810 18:43:05.509150 20889 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a36c9ad578b45af9c80bc9e754b7105 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a36c9ad578b45af9c80bc9e754b7105" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 39209 } }
I20260810 18:43:05.509727 20889 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a36c9ad578b45af9c80bc9e754b7105 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:43:05.509871 20889 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a36c9ad578b45af9c80bc9e754b7105 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a36c9ad578b45af9c80bc9e754b7105, State: Initialized, Role: FOLLOWER
I20260810 18:43:05.510615 20889 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a36c9ad578b45af9c80bc9e754b7105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a36c9ad578b45af9c80bc9e754b7105" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 39209 } }
I20260810 18:43:05.510972 20889 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a36c9ad578b45af9c80bc9e754b7105 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:43:05.511142 20889 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a36c9ad578b45af9c80bc9e754b7105 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:43:05.511484 20889 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a36c9ad578b45af9c80bc9e754b7105 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:43:05.514240 20889 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a36c9ad578b45af9c80bc9e754b7105 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a36c9ad578b45af9c80bc9e754b7105" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 39209 } }
I20260810 18:43:05.514940 20889 leader_election.cc:302] T 00000000000000000000000000000000 P 6a36c9ad578b45af9c80bc9e754b7105 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a36c9ad578b45af9c80bc9e754b7105; no voters:
I20260810 18:43:05.515708 20889 leader_election.cc:288] T 00000000000000000000000000000000 P 6a36c9ad578b45af9c80bc9e754b7105 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:43:05.515928 20896 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a36c9ad578b45af9c80bc9e754b7105 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:43:05.517220 20896 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a36c9ad578b45af9c80bc9e754b7105 [term 1 LEADER]: Becoming Leader. State: Replica: 6a36c9ad578b45af9c80bc9e754b7105, State: Running, Role: LEADER
I20260810 18:43:05.517831 20896 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a36c9ad578b45af9c80bc9e754b7105 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a36c9ad578b45af9c80bc9e754b7105" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 39209 } }
I20260810 18:43:05.518347 20889 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a36c9ad578b45af9c80bc9e754b7105 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:43:05.522650 20897 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a36c9ad578b45af9c80bc9e754b7105 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a36c9ad578b45af9c80bc9e754b7105" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a36c9ad578b45af9c80bc9e754b7105" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 39209 } } }
I20260810 18:43:05.523226 20897 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a36c9ad578b45af9c80bc9e754b7105 [sys.catalog]: This master's current role is: LEADER
I20260810 18:43:05.524171 20898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a36c9ad578b45af9c80bc9e754b7105 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a36c9ad578b45af9c80bc9e754b7105. Latest consensus state: current_term: 1 leader_uuid: "6a36c9ad578b45af9c80bc9e754b7105" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a36c9ad578b45af9c80bc9e754b7105" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 39209 } } }
I20260810 18:43:05.524647 20898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a36c9ad578b45af9c80bc9e754b7105 [sys.catalog]: This master's current role is: LEADER
I20260810 18:43:05.526134 20909 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:43:05.532986 20909 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:43:05.545665 20909 catalog_manager.cc:1347] Generated new cluster ID: 73f4b034107c468d861186e226a90178
I20260810 18:43:05.545877 20909 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:43:05.556138 20909 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:43:05.558611 20909 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:43:05.573583 20909 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a36c9ad578b45af9c80bc9e754b7105: Generated new TSK 0
I20260810 18:43:05.575022 20909 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.228.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab.
Entry for principal kudu/127.4.228.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab.
I20260810 18:43:05.668423 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.193:0
--local_ip_for_outbound_sockets=127.4.228.193
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:39209
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:44041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:43:06.066917 20925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:43:06.067245 20925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:43:06.067426 20925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:43:06.075493 20925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:43:06.075749 20925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:43:06.076085 20925 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.193
I20260810 18:43:06.086679 20925 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:44041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:39209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.193
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:43:06.088212 20925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:43:06.089694 20925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:43:06.104521 20932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:43:06.104509 20933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:43:06.105172 20939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:43:06.105726 20925 server_base.cc:1034] running on GCE node
I20260810 18:43:06.106632 20925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:43:06.107913 20925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:43:06.109185 20925 hybrid_clock.cc:648] HybridClock initialized: now 1786387386109135 us; error 37 us; skew 500 ppm
I20260810 18:43:06.109579 20925 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:43:06 dist-test-slave-7kzw krb5kdc[20753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387386, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:43:06.113541 20925 init.cc:377] Logged in from keytab as kudu/127.4.228.193@KRBTEST.COM (short username kudu)
I20260810 18:43:06.115628 20925 webserver.cc:460] Webserver started at http://127.4.228.193:43907/ using document root <none> and password file <none>
I20260810 18:43:06.116535 20925 fs_manager.cc:359] Metadata directory not provided
I20260810 18:43:06.116717 20925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:43:06.117106 20925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:43:06.120854 20925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/data/instance:
uuid: "ca7f5741d77a4b1b9bf1fdc572b7d8f6"
format_stamp: "Formatted at 2026-08-10 18:43:06 on dist-test-slave-7kzw"
server_key: "01c6718db8ab280f15791143c36cd3ad"
server_key_iv: "1316b119b0226855df163879e8eaf4f2"
server_key_version: "encryptionkey@0"
I20260810 18:43:06.121868 20925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance:
uuid: "ca7f5741d77a4b1b9bf1fdc572b7d8f6"
format_stamp: "Formatted at 2026-08-10 18:43:06 on dist-test-slave-7kzw"
server_key: "01c6718db8ab280f15791143c36cd3ad"
server_key_iv: "1316b119b0226855df163879e8eaf4f2"
server_key_version: "encryptionkey@0"
I20260810 18:43:06.128994 20925 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260810 18:43:06.132978 20950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:43:06.134773 20925 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:43:06.135000 20925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/wal
uuid: "ca7f5741d77a4b1b9bf1fdc572b7d8f6"
format_stamp: "Formatted at 2026-08-10 18:43:06 on dist-test-slave-7kzw"
server_key: "01c6718db8ab280f15791143c36cd3ad"
server_key_iv: "1316b119b0226855df163879e8eaf4f2"
server_key_version: "encryptionkey@0"
I20260810 18:43:06.135267 20925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:43:06.151417 20925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:43:06.156418 20925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:43:06.156770 20925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:43:06.157928 20925 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:43:06.159904 20925 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:43:06.160043 20925 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:43:06.160252 20925 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:43:06.160341 20925 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:43:06.201189 20925 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.193:43887
I20260810 18:43:06.201192 21113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.193:43887 every 8 connection(s)
I20260810 18:43:06.202800 20925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
I20260810 18:43:06.205312 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 20925
I20260810 18:43:06.205549 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance
I20260810 18:43:06.206527 5011 external_mini_cluster.cc:1428] Setting key 2bec5ba7928102253f533b69e946f987
Aug 10 18:43:06 dist-test-slave-7kzw krb5kdc[20753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387386, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:43:06.228436 21118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:43:06.206884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:43883 (local address 127.4.228.254:39209)
0810 18:43:06.207542 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0810 18:43:06.207552 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:43:06.208168 (+ 616us) server_negotiation.cc:407] Connection header received
0810 18:43:06.210335 (+ 2167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:43:06.210344 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:43:06.210451 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:43:06.210619 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:43:06.212409 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:43:06.213609 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:43:06.215634 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:43:06.216360 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:43:06.216538 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:43:06.220742 (+ 4204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:43:06.220751 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:43:06.220757 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:43:06.220810 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:43:06.224114 (+ 3304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:43:06.225405 (+ 1291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:43:06.225411 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:43:06.225413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:43:06.225562 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:43:06.226185 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:43:06.226189 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:43:06.226191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:43:06.226568 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:43:06.226718 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 18:43:06.227836 (+ 1118us) server_negotiation.cc:299] Negotiation successful
0810 18:43:06.228129 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":330,"threads_started":1}
I20260810 18:43:06.229995 21123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:43:06.206729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:39209 (local address 127.4.228.193:43883)
0810 18:43:06.207947 (+ 1218us) negotiation.cc:107] Waiting for socket to connect
0810 18:43:06.208019 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0810 18:43:06.210007 (+ 1988us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:43:06.210936 (+ 929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:43:06.210945 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:43:06.211545 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:43:06.212187 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:43:06.212206 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:43:06.213841 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:43:06.213847 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:43:06.215428 (+ 1581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:43:06.215445 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:43:06.216699 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:43:06.216706 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:43:06.216957 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:43:06.217827 (+ 870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:43:06.217854 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:43:06.220473 (+ 2619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:43:06.224320 (+ 3847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:43:06.224334 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:43:06.224356 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:43:06.225140 (+ 784us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:43:06.225753 (+ 613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:43:06.225761 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:43:06.225768 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:43:06.226058 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:43:06.226725 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:43:06.226738 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:43:06.227321 (+ 583us) client_negotiation.cc:769] Sending connection context
0810 18:43:06.227691 (+ 370us) client_negotiation.cc:240] Negotiation successful
0810 18:43:06.228331 (+ 640us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":898,"thread_start_us":633,"threads_started":1}
I20260810 18:43:06.231928 21115 heartbeater.cc:347] Connected to a master server at 127.4.228.254:39209
I20260810 18:43:06.232348 21115 heartbeater.cc:464] Registering TS with master...
I20260810 18:43:06.233385 21115 heartbeater.cc:511] Master 127.4.228.254:39209 requested a full tablet report, sending...
I20260810 18:43:06.236101 20836 ts_manager.cc:194] Registered new tserver with Master: ca7f5741d77a4b1b9bf1fdc572b7d8f6 (127.4.228.193:43887)
I20260810 18:43:06.237700 20836 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.193@KRBTEST.COM'} at 127.4.228.193:43883
WARNING: no policy specified for kudu/127.4.228.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab.
Entry for principal kudu/127.4.228.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab.
I20260810 18:43:06.283623 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.194:0
--local_ip_for_outbound_sockets=127.4.228.194
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:39209
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:44041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:43:06.660312 21127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:43:06.660770 21127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:43:06.660943 21127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:43:06.669914 21127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:43:06.670081 21127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:43:06.670274 21127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.194
I20260810 18:43:06.680711 21127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:44041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:39209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.194
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:43:06.682214 21127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:43:06.683745 21127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:43:06.698406 21138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:43:06.698504 21135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:43:06.698452 21134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:43:06.699502 21127 server_base.cc:1034] running on GCE node
I20260810 18:43:06.700209 21127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:43:06.701294 21127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:43:06.702574 21127 hybrid_clock.cc:648] HybridClock initialized: now 1786387386702524 us; error 37 us; skew 500 ppm
I20260810 18:43:06.702931 21127 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:43:06 dist-test-slave-7kzw krb5kdc[20753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387386, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:43:06.706259 21127 init.cc:377] Logged in from keytab as kudu/127.4.228.194@KRBTEST.COM (short username kudu)
I20260810 18:43:06.707947 21127 webserver.cc:460] Webserver started at http://127.4.228.194:38521/ using document root <none> and password file <none>
I20260810 18:43:06.708670 21127 fs_manager.cc:359] Metadata directory not provided
I20260810 18:43:06.708781 21127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:43:06.709048 21127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:43:06.712280 21127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/data/instance:
uuid: "32a28614ccb146f29e0fb95dd7f9c8fa"
format_stamp: "Formatted at 2026-08-10 18:43:06 on dist-test-slave-7kzw"
server_key: "6d63edadd00567d1e072e1c7b61e24a5"
server_key_iv: "b7c61763929d16f7d43a80cabe909fa2"
server_key_version: "encryptionkey@0"
I20260810 18:43:06.713088 21127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance:
uuid: "32a28614ccb146f29e0fb95dd7f9c8fa"
format_stamp: "Formatted at 2026-08-10 18:43:06 on dist-test-slave-7kzw"
server_key: "6d63edadd00567d1e072e1c7b61e24a5"
server_key_iv: "b7c61763929d16f7d43a80cabe909fa2"
server_key_version: "encryptionkey@0"
I20260810 18:43:06.718968 21127 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260810 18:43:06.723001 21152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:43:06.724850 21127 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260810 18:43:06.725019 21127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/wal
uuid: "32a28614ccb146f29e0fb95dd7f9c8fa"
format_stamp: "Formatted at 2026-08-10 18:43:06 on dist-test-slave-7kzw"
server_key: "6d63edadd00567d1e072e1c7b61e24a5"
server_key_iv: "b7c61763929d16f7d43a80cabe909fa2"
server_key_version: "encryptionkey@0"
I20260810 18:43:06.725296 21127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:43:06.771513 21127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:43:06.776710 21127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:43:06.777117 21127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:43:06.778324 21127 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:43:06.780032 21127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:43:06.780160 21127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:43:06.780333 21127 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:43:06.780416 21127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:43:06.819837 21127 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.194:44293
I20260810 18:43:06.819937 21314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.194:44293 every 8 connection(s)
I20260810 18:43:06.821338 21127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
I20260810 18:43:06.828047 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 21127
I20260810 18:43:06.828266 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance
I20260810 18:43:06.828877 5011 external_mini_cluster.cc:1428] Setting key 4749c787fa2f4dfbca58cbed9c340e8f
Aug 10 18:43:06 dist-test-slave-7kzw krb5kdc[20753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387386, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:43:06.846294 21318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:43:06.824451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:54633 (local address 127.4.228.254:39209)
0810 18:43:06.825006 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0810 18:43:06.825013 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:43:06.826032 (+ 1019us) server_negotiation.cc:407] Connection header received
0810 18:43:06.828294 (+ 2262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:43:06.828302 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:43:06.828402 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:43:06.828534 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:43:06.830537 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:43:06.831526 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:43:06.833237 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:43:06.834035 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:43:06.834198 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:43:06.838071 (+ 3873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:43:06.838081 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:43:06.838089 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:43:06.838139 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:43:06.842276 (+ 4137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:43:06.843302 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:43:06.843310 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:43:06.843316 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:43:06.843506 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:43:06.844271 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:43:06.844277 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:43:06.844279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:43:06.844632 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:43:06.844841 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0810 18:43:06.845813 (+ 972us) server_negotiation.cc:299] Negotiation successful
0810 18:43:06.846015 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":279,"threads_started":1}
I20260810 18:43:06.848160 21319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:43:06.825000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:39209 (local address 127.4.228.194:54633)
0810 18:43:06.825862 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0810 18:43:06.825905 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 18:43:06.827901 (+ 1996us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:43:06.828783 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:43:06.828793 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:43:06.829613 (+ 820us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:43:06.830344 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:43:06.830364 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:43:06.831697 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:43:06.831702 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:43:06.833066 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:43:06.833081 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:43:06.834270 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:43:06.834275 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:43:06.834549 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:43:06.835301 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:43:06.835324 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:43:06.837797 (+ 2473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:43:06.842572 (+ 4775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:43:06.842583 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:43:06.842601 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:43:06.843147 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:43:06.843678 (+ 531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:43:06.843682 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:43:06.843685 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:43:06.844052 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:43:06.844817 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:43:06.844827 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:43:06.845278 (+ 451us) client_negotiation.cc:769] Sending connection context
0810 18:43:06.845668 (+ 390us) client_negotiation.cc:240] Negotiation successful
0810 18:43:06.846372 (+ 704us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":341,"threads_started":1}
I20260810 18:43:06.850035 21315 heartbeater.cc:347] Connected to a master server at 127.4.228.254:39209
I20260810 18:43:06.850404 21315 heartbeater.cc:464] Registering TS with master...
I20260810 18:43:06.851291 21315 heartbeater.cc:511] Master 127.4.228.254:39209 requested a full tablet report, sending...
I20260810 18:43:06.853071 20836 ts_manager.cc:194] Registered new tserver with Master: 32a28614ccb146f29e0fb95dd7f9c8fa (127.4.228.194:44293)
I20260810 18:43:06.853960 20836 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.194@KRBTEST.COM'} at 127.4.228.194:54633
WARNING: no policy specified for kudu/127.4.228.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab.
Entry for principal kudu/127.4.228.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab.
I20260810 18:43:06.904657 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.195:0
--local_ip_for_outbound_sockets=127.4.228.195
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:39209
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:44041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:43:07.241899 21115 heartbeater.cc:503] Master 127.4.228.254:39209 was elected leader, sending a full tablet report...
W20260810 18:43:07.286113 21328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:43:07.286548 21328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:43:07.286725 21328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:43:07.294543 21328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:43:07.294709 21328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:43:07.294885 21328 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.195
I20260810 18:43:07.305395 21328 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:44041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:39209
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.195
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:43:07.307264 21328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:43:07.309054 21328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:43:07.324434 21336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:43:07.324432 21340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:43:07.324534 21337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:43:07.325577 21328 server_base.cc:1034] running on GCE node
I20260810 18:43:07.326421 21328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:43:07.327605 21328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:43:07.328866 21328 hybrid_clock.cc:648] HybridClock initialized: now 1786387387328807 us; error 51 us; skew 500 ppm
I20260810 18:43:07.329289 21328 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:43:07 dist-test-slave-7kzw krb5kdc[20753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387387, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:43:07.333019 21328 init.cc:377] Logged in from keytab as kudu/127.4.228.195@KRBTEST.COM (short username kudu)
I20260810 18:43:07.334740 21328 webserver.cc:460] Webserver started at http://127.4.228.195:38557/ using document root <none> and password file <none>
I20260810 18:43:07.335520 21328 fs_manager.cc:359] Metadata directory not provided
I20260810 18:43:07.335690 21328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:43:07.336024 21328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:43:07.339306 21328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/data/instance:
uuid: "7d0120ef53b244c6a1e0e329b7ce23be"
format_stamp: "Formatted at 2026-08-10 18:43:07 on dist-test-slave-7kzw"
server_key: "6f0ae65cdb2900197e14bce1b0541793"
server_key_iv: "c251e33576c5ffd5645b42dd3188d085"
server_key_version: "encryptionkey@0"
I20260810 18:43:07.340242 21328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance:
uuid: "7d0120ef53b244c6a1e0e329b7ce23be"
format_stamp: "Formatted at 2026-08-10 18:43:07 on dist-test-slave-7kzw"
server_key: "6f0ae65cdb2900197e14bce1b0541793"
server_key_iv: "c251e33576c5ffd5645b42dd3188d085"
server_key_version: "encryptionkey@0"
I20260810 18:43:07.345897 21328 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 18:43:07.350067 21347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:43:07.352039 21328 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 18:43:07.352270 21328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/wal
uuid: "7d0120ef53b244c6a1e0e329b7ce23be"
format_stamp: "Formatted at 2026-08-10 18:43:07 on dist-test-slave-7kzw"
server_key: "6f0ae65cdb2900197e14bce1b0541793"
server_key_iv: "c251e33576c5ffd5645b42dd3188d085"
server_key_version: "encryptionkey@0"
I20260810 18:43:07.352501 21328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:43:07.366811 21328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:43:07.372066 21328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:43:07.372428 21328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:43:07.373603 21328 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:43:07.375247 21328 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:43:07.375361 21328 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:43:07.375522 21328 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:43:07.375603 21328 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:43:07.412451 21328 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.195:33547
I20260810 18:43:07.412546 21511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.195:33547 every 8 connection(s)
I20260810 18:43:07.413825 21328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
I20260810 18:43:07.420701 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 21328
I20260810 18:43:07.420935 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance
I20260810 18:43:07.421478 5011 external_mini_cluster.cc:1428] Setting key 4520cc76f1032a33543e96cb9a7e3db9
Aug 10 18:43:07 dist-test-slave-7kzw krb5kdc[20753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387387, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:43:07.437793 21515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:43:07.416776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:57585 (local address 127.4.228.254:39209)
0810 18:43:07.417263 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0810 18:43:07.417268 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:43:07.418421 (+ 1153us) server_negotiation.cc:407] Connection header received
0810 18:43:07.420654 (+ 2233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:43:07.420663 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:43:07.420771 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:43:07.420923 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:43:07.422994 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:43:07.424058 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:43:07.426343 (+ 2285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:43:07.426866 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:43:07.427010 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:43:07.431157 (+ 4147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:43:07.431164 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:43:07.431168 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:43:07.431205 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:43:07.434203 (+ 2998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:43:07.435135 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:43:07.435140 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:43:07.435142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:43:07.435278 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:43:07.435790 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:43:07.435795 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:43:07.435797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:43:07.436102 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:43:07.436299 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0810 18:43:07.437095 (+ 796us) server_negotiation.cc:299] Negotiation successful
0810 18:43:07.437362 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":230,"threads_started":1}
I20260810 18:43:07.439010 21517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:43:07.417327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:39209 (local address 127.4.228.195:57585)
0810 18:43:07.418238 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0810 18:43:07.418280 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 18:43:07.420317 (+ 2037us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:43:07.421241 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:43:07.421250 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:43:07.421937 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:43:07.422795 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:43:07.422817 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:43:07.424243 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:43:07.424247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:43:07.426163 (+ 1916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:43:07.426180 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:43:07.427128 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:43:07.427138 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:43:07.427471 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:43:07.428221 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:43:07.428246 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:43:07.430902 (+ 2656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:43:07.434385 (+ 3483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:43:07.434397 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:43:07.434424 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:43:07.434969 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:43:07.435408 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:43:07.435413 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:43:07.435418 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:43:07.435648 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:43:07.436261 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:43:07.436270 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:43:07.436668 (+ 398us) client_negotiation.cc:769] Sending connection context
0810 18:43:07.436960 (+ 292us) client_negotiation.cc:240] Negotiation successful
0810 18:43:07.437564 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":349,"threads_started":1}
I20260810 18:43:07.440587 21512 heartbeater.cc:347] Connected to a master server at 127.4.228.254:39209
I20260810 18:43:07.440945 21512 heartbeater.cc:464] Registering TS with master...
I20260810 18:43:07.441735 21512 heartbeater.cc:511] Master 127.4.228.254:39209 requested a full tablet report, sending...
I20260810 18:43:07.443405 20836 ts_manager.cc:194] Registered new tserver with Master: 7d0120ef53b244c6a1e0e329b7ce23be (127.4.228.195:33547)
I20260810 18:43:07.444269 20836 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.195@KRBTEST.COM'} at 127.4.228.195:57585
I20260810 18:43:07.450333 5011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:43:07.468896 21515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:43:07.455460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40252 (local address 127.4.228.254:39209)
0810 18:43:07.455634 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 18:43:07.455639 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:43:07.456260 (+ 621us) server_negotiation.cc:407] Connection header received
0810 18:43:07.456476 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:43:07.456480 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:43:07.456560 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:43:07.456664 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:43:07.458254 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:43:07.459071 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:43:07.460317 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:43:07.460722 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:43:07.460857 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:43:07.463319 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:43:07.463325 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:43:07.463328 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:43:07.463364 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:43:07.465956 (+ 2592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:43:07.466714 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:43:07.466722 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:43:07.466726 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:43:07.466870 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:43:07.467317 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:43:07.467322 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:43:07.467328 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:43:07.467678 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:43:07.467942 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0810 18:43:07.468315 (+ 373us) server_negotiation.cc:299] Negotiation successful
0810 18:43:07.468545 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 18:43:07.474221 20836 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40252:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:43:07.485380 20836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:43:07.519637 21537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:43:07.499890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:33547 (local address 127.0.0.1:44118)
0810 18:43:07.508054 (+ 8164us) negotiation.cc:107] Waiting for socket to connect
0810 18:43:07.508086 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 18:43:07.508225 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:43:07.510237 (+ 2012us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:43:07.510242 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:43:07.510278 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:43:07.510969 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:43:07.510986 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:43:07.513196 (+ 2210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:43:07.513202 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:43:07.516022 (+ 2820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:43:07.516039 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:43:07.518531 (+ 2492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:43:07.518539 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:43:07.518768 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:43:07.518809 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 18:43:07.518885 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 18:43:07.519064 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7999,"mutex_wait_us":109,"thread_start_us":7820,"threads_started":1}
I20260810 18:43:07.520243 21533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:43:07.499478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:43887 (local address 127.0.0.1:60702)
0810 18:43:07.505935 (+ 6457us) negotiation.cc:107] Waiting for socket to connect
0810 18:43:07.505986 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 18:43:07.506181 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:43:07.509375 (+ 3194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:43:07.509385 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:43:07.509433 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:43:07.510446 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:43:07.510488 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:43:07.512652 (+ 2164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:43:07.512659 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:43:07.515433 (+ 2774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:43:07.515451 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:43:07.517983 (+ 2532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:43:07.517992 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:43:07.518208 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:43:07.518288 (+ 80us) client_negotiation.cc:769] Sending connection context
0810 18:43:07.518564 (+ 276us) client_negotiation.cc:240] Negotiation successful
0810 18:43:07.519681 (+ 1117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6176,"mutex_wait_us":7,"thread_start_us":5999,"threads_started":1}
I20260810 18:43:07.519481 21536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:43:07.499760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60702 (local address 127.4.228.193:43887)
0810 18:43:07.508501 (+ 8741us) server_negotiation.cc:206] Beginning negotiation
0810 18:43:07.508514 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:43:07.508553 (+ 39us) server_negotiation.cc:407] Connection header received
0810 18:43:07.508675 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:43:07.508684 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:43:07.509078 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:43:07.509783 (+ 705us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:43:07.510865 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:43:07.512483 (+ 1618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:43:07.515651 (+ 3168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:43:07.517333 (+ 1682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:43:07.517507 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:43:07.517893 (+ 386us) server_negotiation.cc:1035] Waiting for connection context
0810 18:43:07.518834 (+ 941us) server_negotiation.cc:299] Negotiation successful
0810 18:43:07.519051 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8511,"thread_start_us":8323,"threads_started":1}
I20260810 18:43:07.520519 21540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:43:07.504878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48830 (local address 127.4.228.194:44293)
0810 18:43:07.509280 (+ 4402us) server_negotiation.cc:206] Beginning negotiation
0810 18:43:07.509292 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:43:07.511326 (+ 2034us) server_negotiation.cc:407] Connection header received
0810 18:43:07.511530 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:43:07.511544 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:43:07.511838 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:43:07.512066 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:43:07.512705 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:43:07.514338 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:43:07.516984 (+ 2646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:43:07.518646 (+ 1662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:43:07.518813 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:43:07.519025 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0810 18:43:07.519883 (+ 858us) server_negotiation.cc:299] Negotiation successful
0810 18:43:07.520094 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4202,"thread_start_us":4079,"threads_started":1}
I20260810 18:43:07.521055 21539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:43:07.504206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:44293 (local address 127.0.0.1:48830)
0810 18:43:07.511159 (+ 6953us) negotiation.cc:107] Waiting for socket to connect
0810 18:43:07.511178 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:43:07.511303 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:43:07.512081 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:43:07.512086 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:43:07.512115 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:43:07.512491 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:43:07.512502 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:43:07.514571 (+ 2069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:43:07.514583 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:43:07.516715 (+ 2132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:43:07.516768 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:43:07.519214 (+ 2446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:43:07.519226 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:43:07.519417 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:43:07.519464 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 18:43:07.519576 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 18:43:07.519705 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5564,"mutex_wait_us":2,"thread_start_us":5376,"threads_started":1}
I20260810 18:43:07.521720 21538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:43:07.500292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44118 (local address 127.4.228.195:33547)
0810 18:43:07.508894 (+ 8602us) server_negotiation.cc:206] Beginning negotiation
0810 18:43:07.508905 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:43:07.508940 (+ 35us) server_negotiation.cc:407] Connection header received
0810 18:43:07.509051 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:43:07.509059 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:43:07.509323 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:43:07.510270 (+ 947us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:43:07.511216 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:43:07.513020 (+ 1804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:43:07.516233 (+ 3213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:43:07.518209 (+ 1976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:43:07.520245 (+ 2036us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:43:07.520924 (+ 679us) server_negotiation.cc:1035] Waiting for connection context
0810 18:43:07.521021 (+ 97us) server_negotiation.cc:299] Negotiation successful
0810 18:43:07.521431 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8413,"thread_start_us":8226,"threads_started":1}
I20260810 18:43:07.525413 21017 tablet_service.cc:1467] Processing CreateTablet for tablet e36f01e2c74346e0b7738f21716d2e65 (DEFAULT_TABLE table=test-table [id=9fc50163adc84f68af950f4161ed4a83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:43:07.525426 21226 tablet_service.cc:1467] Processing CreateTablet for tablet e36f01e2c74346e0b7738f21716d2e65 (DEFAULT_TABLE table=test-table [id=9fc50163adc84f68af950f4161ed4a83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:43:07.526036 21420 tablet_service.cc:1467] Processing CreateTablet for tablet e36f01e2c74346e0b7738f21716d2e65 (DEFAULT_TABLE table=test-table [id=9fc50163adc84f68af950f4161ed4a83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:43:07.527082 21226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e36f01e2c74346e0b7738f21716d2e65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:43:07.527098 21017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e36f01e2c74346e0b7738f21716d2e65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:43:07.527532 21420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e36f01e2c74346e0b7738f21716d2e65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:43:07.541460 21543 tablet_bootstrap.cc:492] T e36f01e2c74346e0b7738f21716d2e65 P ca7f5741d77a4b1b9bf1fdc572b7d8f6: Bootstrap starting.
I20260810 18:43:07.544088 21544 tablet_bootstrap.cc:492] T e36f01e2c74346e0b7738f21716d2e65 P 32a28614ccb146f29e0fb95dd7f9c8fa: Bootstrap starting.
I20260810 18:43:07.546017 21547 tablet_bootstrap.cc:492] T e36f01e2c74346e0b7738f21716d2e65 P 7d0120ef53b244c6a1e0e329b7ce23be: Bootstrap starting.
I20260810 18:43:07.546692 21543 tablet_bootstrap.cc:654] T e36f01e2c74346e0b7738f21716d2e65 P ca7f5741d77a4b1b9bf1fdc572b7d8f6: Neither blocks nor log segments found. Creating new log.
I20260810 18:43:07.548446 21544 tablet_bootstrap.cc:654] T e36f01e2c74346e0b7738f21716d2e65 P 32a28614ccb146f29e0fb95dd7f9c8fa: Neither blocks nor log segments found. Creating new log.
I20260810 18:43:07.548986 21543 log.cc:824] T e36f01e2c74346e0b7738f21716d2e65 P ca7f5741d77a4b1b9bf1fdc572b7d8f6: Log is configured to *not* fsync() on all Append() calls
I20260810 18:43:07.549639 21547 tablet_bootstrap.cc:654] T e36f01e2c74346e0b7738f21716d2e65 P 7d0120ef53b244c6a1e0e329b7ce23be: Neither blocks nor log segments found. Creating new log.
I20260810 18:43:07.549937 21544 log.cc:824] T e36f01e2c74346e0b7738f21716d2e65 P 32a28614ccb146f29e0fb95dd7f9c8fa: Log is configured to *not* fsync() on all Append() calls
I20260810 18:43:07.550932 21547 log.cc:824] T e36f01e2c74346e0b7738f21716d2e65 P 7d0120ef53b244c6a1e0e329b7ce23be: Log is configured to *not* fsync() on all Append() calls
I20260810 18:43:07.552958 21543 tablet_bootstrap.cc:492] T e36f01e2c74346e0b7738f21716d2e65 P ca7f5741d77a4b1b9bf1fdc572b7d8f6: No bootstrap required, opened a new log
I20260810 18:43:07.553364 21543 ts_tablet_manager.cc:1397] T e36f01e2c74346e0b7738f21716d2e65 P ca7f5741d77a4b1b9bf1fdc572b7d8f6: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.005s
I20260810 18:43:07.554047 21547 tablet_bootstrap.cc:492] T e36f01e2c74346e0b7738f21716d2e65 P 7d0120ef53b244c6a1e0e329b7ce23be: No bootstrap required, opened a new log
I20260810 18:43:07.554047 21544 tablet_bootstrap.cc:492] T e36f01e2c74346e0b7738f21716d2e65 P 32a28614ccb146f29e0fb95dd7f9c8fa: No bootstrap required, opened a new log
I20260810 18:43:07.554324 21544 ts_tablet_manager.cc:1397] T e36f01e2c74346e0b7738f21716d2e65 P 32a28614ccb146f29e0fb95dd7f9c8fa: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.004s
I20260810 18:43:07.554324 21547 ts_tablet_manager.cc:1397] T e36f01e2c74346e0b7738f21716d2e65 P 7d0120ef53b244c6a1e0e329b7ce23be: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 18:43:07.562433 21543 raft_consensus.cc:348] T e36f01e2c74346e0b7738f21716d2e65 P ca7f5741d77a4b1b9bf1fdc572b7d8f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca7f5741d77a4b1b9bf1fdc572b7d8f6" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43887 } } peers { permanent_uuid: "7d0120ef53b244c6a1e0e329b7ce23be" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 33547 } } peers { permanent_uuid: "32a28614ccb146f29e0fb95dd7f9c8fa" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44293 } }
I20260810 18:43:07.563002 21543 raft_consensus.cc:374] T e36f01e2c74346e0b7738f21716d2e65 P ca7f5741d77a4b1b9bf1fdc572b7d8f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:43:07.563117 21543 raft_consensus.cc:729] T e36f01e2c74346e0b7738f21716d2e65 P ca7f5741d77a4b1b9bf1fdc572b7d8f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca7f5741d77a4b1b9bf1fdc572b7d8f6, State: Initialized, Role: FOLLOWER
I20260810 18:43:07.563880 21543 consensus_queue.cc:260] T e36f01e2c74346e0b7738f21716d2e65 P ca7f5741d77a4b1b9bf1fdc572b7d8f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca7f5741d77a4b1b9bf1fdc572b7d8f6" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43887 } } peers { permanent_uuid: "7d0120ef53b244c6a1e0e329b7ce23be" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 33547 } } peers { permanent_uuid: "32a28614ccb146f29e0fb95dd7f9c8fa" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44293 } }
I20260810 18:43:07.565547 21547 raft_consensus.cc:348] T e36f01e2c74346e0b7738f21716d2e65 P 7d0120ef53b244c6a1e0e329b7ce23be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca7f5741d77a4b1b9bf1fdc572b7d8f6" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43887 } } peers { permanent_uuid: "7d0120ef53b244c6a1e0e329b7ce23be" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 33547 } } peers { permanent_uuid: "32a28614ccb146f29e0fb95dd7f9c8fa" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44293 } }
I20260810 18:43:07.565547 21544 raft_consensus.cc:348] T e36f01e2c74346e0b7738f21716d2e65 P 32a28614ccb146f29e0fb95dd7f9c8fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca7f5741d77a4b1b9bf1fdc572b7d8f6" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43887 } } peers { permanent_uuid: "7d0120ef53b244c6a1e0e329b7ce23be" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 33547 } } peers { permanent_uuid: "32a28614ccb146f29e0fb95dd7f9c8fa" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44293 } }
I20260810 18:43:07.566022 21544 raft_consensus.cc:374] T e36f01e2c74346e0b7738f21716d2e65 P 32a28614ccb146f29e0fb95dd7f9c8fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:43:07.566023 21547 raft_consensus.cc:374] T e36f01e2c74346e0b7738f21716d2e65 P 7d0120ef53b244c6a1e0e329b7ce23be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:43:07.566071 21543 ts_tablet_manager.cc:1428] T e36f01e2c74346e0b7738f21716d2e65 P ca7f5741d77a4b1b9bf1fdc572b7d8f6: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
I20260810 18:43:07.566123 21547 raft_consensus.cc:729] T e36f01e2c74346e0b7738f21716d2e65 P 7d0120ef53b244c6a1e0e329b7ce23be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d0120ef53b244c6a1e0e329b7ce23be, State: Initialized, Role: FOLLOWER
I20260810 18:43:07.566123 21544 raft_consensus.cc:729] T e36f01e2c74346e0b7738f21716d2e65 P 32a28614ccb146f29e0fb95dd7f9c8fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32a28614ccb146f29e0fb95dd7f9c8fa, State: Initialized, Role: FOLLOWER
I20260810 18:43:07.566826 21547 consensus_queue.cc:260] T e36f01e2c74346e0b7738f21716d2e65 P 7d0120ef53b244c6a1e0e329b7ce23be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca7f5741d77a4b1b9bf1fdc572b7d8f6" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43887 } } peers { permanent_uuid: "7d0120ef53b244c6a1e0e329b7ce23be" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 33547 } } peers { permanent_uuid: "32a28614ccb146f29e0fb95dd7f9c8fa" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44293 } }
I20260810 18:43:07.566824 21544 consensus_queue.cc:260] T e36f01e2c74346e0b7738f21716d2e65 P 32a28614ccb146f29e0fb95dd7f9c8fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca7f5741d77a4b1b9bf1fdc572b7d8f6" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43887 } } peers { permanent_uuid: "7d0120ef53b244c6a1e0e329b7ce23be" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 33547 } } peers { permanent_uuid: "32a28614ccb146f29e0fb95dd7f9c8fa" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44293 } }
I20260810 18:43:07.568598 21512 heartbeater.cc:503] Master 127.4.228.254:39209 was elected leader, sending a full tablet report...
I20260810 18:43:07.568686 21315 heartbeater.cc:503] Master 127.4.228.254:39209 was elected leader, sending a full tablet report...
I20260810 18:43:07.568912 21544 ts_tablet_manager.cc:1428] T e36f01e2c74346e0b7738f21716d2e65 P 32a28614ccb146f29e0fb95dd7f9c8fa: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
I20260810 18:43:07.570598 21547 ts_tablet_manager.cc:1428] T e36f01e2c74346e0b7738f21716d2e65 P 7d0120ef53b244c6a1e0e329b7ce23be: Time spent starting tablet: real 0.016s user 0.017s sys 0.000s
W20260810 18:43:07.573668 21316 tablet.cc:2369] T e36f01e2c74346e0b7738f21716d2e65 P 32a28614ccb146f29e0fb95dd7f9c8fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:43:07.665860 21513 tablet.cc:2369] T e36f01e2c74346e0b7738f21716d2e65 P 7d0120ef53b244c6a1e0e329b7ce23be: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:43:07.705711 21116 tablet.cc:2369] T e36f01e2c74346e0b7738f21716d2e65 P ca7f5741d77a4b1b9bf1fdc572b7d8f6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:43:07.717530 21557 raft_consensus.cc:482] T e36f01e2c74346e0b7738f21716d2e65 P 7d0120ef53b244c6a1e0e329b7ce23be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:43:07.718065 21557 raft_consensus.cc:504] T e36f01e2c74346e0b7738f21716d2e65 P 7d0120ef53b244c6a1e0e329b7ce23be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca7f5741d77a4b1b9bf1fdc572b7d8f6" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43887 } } peers { permanent_uuid: "7d0120ef53b244c6a1e0e329b7ce23be" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 33547 } } peers { permanent_uuid: "32a28614ccb146f29e0fb95dd7f9c8fa" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44293 } }
I20260810 18:43:07.720295 21557 leader_election.cc:288] T e36f01e2c74346e0b7738f21716d2e65 P 7d0120ef53b244c6a1e0e329b7ce23be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca7f5741d77a4b1b9bf1fdc572b7d8f6 (127.4.228.193:43887), 32a28614ccb146f29e0fb95dd7f9c8fa (127.4.228.194:44293)
I20260810 18:43:07.727383 21540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:43:07.720853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:45695 (local address 127.4.228.194:44293)
0810 18:43:07.721122 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0810 18:43:07.721134 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:43:07.721738 (+ 604us) server_negotiation.cc:407] Connection header received
0810 18:43:07.721936 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:43:07.721941 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:43:07.722002 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:43:07.722103 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:43:07.722673 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:43:07.723374 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:43:07.725254 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:43:07.726268 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:43:07.726470 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:43:07.726578 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 18:43:07.726932 (+ 354us) server_negotiation.cc:299] Negotiation successful
0810 18:43:07.727064 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 18:43:07.727442 21536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:43:07.720846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:53655 (local address 127.4.228.193:43887)
0810 18:43:07.721124 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0810 18:43:07.721132 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:43:07.721268 (+ 136us) server_negotiation.cc:407] Connection header received
0810 18:43:07.721534 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:43:07.721541 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:43:07.721634 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:43:07.721757 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:43:07.722206 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:43:07.723105 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:43:07.725050 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:43:07.726265 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:43:07.726472 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:43:07.726589 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 18:43:07.726985 (+ 396us) server_negotiation.cc:299] Negotiation successful
0810 18:43:07.727148 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142}
I20260810 18:43:07.727530 21558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:43:07.720784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:44293 (local address 127.4.228.195:45695)
0810 18:43:07.721617 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0810 18:43:07.721638 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:43:07.721791 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:43:07.722133 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:43:07.722138 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:43:07.722161 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:43:07.722511 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:43:07.722525 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:43:07.723533 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:43:07.723543 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:43:07.725102 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:43:07.725116 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:43:07.726521 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:43:07.726528 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:43:07.726764 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:43:07.726791 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 18:43:07.726902 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 18:43:07.727040 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":729,"mutex_wait_us":63,"thread_start_us":498,"threads_started":1}
I20260810 18:43:07.727577 21517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:43:07.720758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:43887 (local address 127.4.228.195:53655)
0810 18:43:07.721112 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0810 18:43:07.721144 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 18:43:07.721314 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:43:07.721789 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:43:07.721795 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:43:07.721828 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:43:07.722047 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:43:07.722058 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:43:07.723260 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:43:07.723268 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:43:07.724885 (+ 1617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:43:07.724901 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:43:07.726516 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:43:07.726523 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:43:07.726767 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:43:07.726820 (+ 53us) client_negotiation.cc:769] Sending connection context
0810 18:43:07.726953 (+ 133us) client_negotiation.cc:240] Negotiation successful
0810 18:43:07.727105 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":7}
I20260810 18:43:07.728906 21049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e36f01e2c74346e0b7738f21716d2e65" candidate_uuid: "7d0120ef53b244c6a1e0e329b7ce23be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca7f5741d77a4b1b9bf1fdc572b7d8f6" is_pre_election: true
I20260810 18:43:07.728965 21257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e36f01e2c74346e0b7738f21716d2e65" candidate_uuid: "7d0120ef53b244c6a1e0e329b7ce23be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32a28614ccb146f29e0fb95dd7f9c8fa" is_pre_election: true
I20260810 18:43:07.729552 21257 raft_consensus.cc:2454] T e36f01e2c74346e0b7738f21716d2e65 P 32a28614ccb146f29e0fb95dd7f9c8fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d0120ef53b244c6a1e0e329b7ce23be in term 0.
I20260810 18:43:07.729583 21049 raft_consensus.cc:2454] T e36f01e2c74346e0b7738f21716d2e65 P ca7f5741d77a4b1b9bf1fdc572b7d8f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d0120ef53b244c6a1e0e329b7ce23be in term 0.
I20260810 18:43:07.730508 21349 leader_election.cc:302] T e36f01e2c74346e0b7738f21716d2e65 P 7d0120ef53b244c6a1e0e329b7ce23be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32a28614ccb146f29e0fb95dd7f9c8fa, 7d0120ef53b244c6a1e0e329b7ce23be; no voters:
I20260810 18:43:07.731032 21557 raft_consensus.cc:2781] T e36f01e2c74346e0b7738f21716d2e65 P 7d0120ef53b244c6a1e0e329b7ce23be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:43:07.731235 21557 raft_consensus.cc:482] T e36f01e2c74346e0b7738f21716d2e65 P 7d0120ef53b244c6a1e0e329b7ce23be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:43:07.731357 21557 raft_consensus.cc:3037] T e36f01e2c74346e0b7738f21716d2e65 P 7d0120ef53b244c6a1e0e329b7ce23be [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:43:07.733644 21557 raft_consensus.cc:504] T e36f01e2c74346e0b7738f21716d2e65 P 7d0120ef53b244c6a1e0e329b7ce23be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca7f5741d77a4b1b9bf1fdc572b7d8f6" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43887 } } peers { permanent_uuid: "7d0120ef53b244c6a1e0e329b7ce23be" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 33547 } } peers { permanent_uuid: "32a28614ccb146f29e0fb95dd7f9c8fa" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44293 } }
I20260810 18:43:07.734422 21557 leader_election.cc:288] T e36f01e2c74346e0b7738f21716d2e65 P 7d0120ef53b244c6a1e0e329b7ce23be [CANDIDATE]: Term 1 election: Requested vote from peers ca7f5741d77a4b1b9bf1fdc572b7d8f6 (127.4.228.193:43887), 32a28614ccb146f29e0fb95dd7f9c8fa (127.4.228.194:44293)
I20260810 18:43:07.734915 21049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e36f01e2c74346e0b7738f21716d2e65" candidate_uuid: "7d0120ef53b244c6a1e0e329b7ce23be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca7f5741d77a4b1b9bf1fdc572b7d8f6"
I20260810 18:43:07.735047 21257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e36f01e2c74346e0b7738f21716d2e65" candidate_uuid: "7d0120ef53b244c6a1e0e329b7ce23be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32a28614ccb146f29e0fb95dd7f9c8fa"
I20260810 18:43:07.735172 21049 raft_consensus.cc:3037] T e36f01e2c74346e0b7738f21716d2e65 P ca7f5741d77a4b1b9bf1fdc572b7d8f6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:43:07.735297 21257 raft_consensus.cc:3037] T e36f01e2c74346e0b7738f21716d2e65 P 32a28614ccb146f29e0fb95dd7f9c8fa [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:43:07.738549 21049 raft_consensus.cc:2454] T e36f01e2c74346e0b7738f21716d2e65 P ca7f5741d77a4b1b9bf1fdc572b7d8f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d0120ef53b244c6a1e0e329b7ce23be in term 1.
I20260810 18:43:07.738549 21257 raft_consensus.cc:2454] T e36f01e2c74346e0b7738f21716d2e65 P 32a28614ccb146f29e0fb95dd7f9c8fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d0120ef53b244c6a1e0e329b7ce23be in term 1.
I20260810 18:43:07.739112 21350 leader_election.cc:302] T e36f01e2c74346e0b7738f21716d2e65 P 7d0120ef53b244c6a1e0e329b7ce23be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d0120ef53b244c6a1e0e329b7ce23be, ca7f5741d77a4b1b9bf1fdc572b7d8f6; no voters:
I20260810 18:43:07.739481 21557 raft_consensus.cc:2781] T e36f01e2c74346e0b7738f21716d2e65 P 7d0120ef53b244c6a1e0e329b7ce23be [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:43:07.740065 21557 raft_consensus.cc:686] T e36f01e2c74346e0b7738f21716d2e65 P 7d0120ef53b244c6a1e0e329b7ce23be [term 1 LEADER]: Becoming Leader. State: Replica: 7d0120ef53b244c6a1e0e329b7ce23be, State: Running, Role: LEADER
I20260810 18:43:07.740701 21557 consensus_queue.cc:237] T e36f01e2c74346e0b7738f21716d2e65 P 7d0120ef53b244c6a1e0e329b7ce23be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca7f5741d77a4b1b9bf1fdc572b7d8f6" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43887 } } peers { permanent_uuid: "7d0120ef53b244c6a1e0e329b7ce23be" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 33547 } } peers { permanent_uuid: "32a28614ccb146f29e0fb95dd7f9c8fa" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44293 } }
I20260810 18:43:07.745692 20831 catalog_manager.cc:5495] T e36f01e2c74346e0b7738f21716d2e65 P 7d0120ef53b244c6a1e0e329b7ce23be reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d0120ef53b244c6a1e0e329b7ce23be (127.4.228.195). New cstate: current_term: 1 leader_uuid: "7d0120ef53b244c6a1e0e329b7ce23be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca7f5741d77a4b1b9bf1fdc572b7d8f6" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d0120ef53b244c6a1e0e329b7ce23be" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 33547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32a28614ccb146f29e0fb95dd7f9c8fa" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44293 } health_report { overall_health: UNKNOWN } } }
I20260810 18:43:07.768674 21538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:43:07.762822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44134 (local address 127.4.228.195:33547)
0810 18:43:07.763041 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 18:43:07.763046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:43:07.763065 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:43:07.763181 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:43:07.763185 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:43:07.763267 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:43:07.763409 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:43:07.763895 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:43:07.764745 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:43:07.766162 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:43:07.766715 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:43:07.766820 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:43:07.767470 (+ 650us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:43:07.768046 (+ 576us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:43:07.768174 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 18:43:07.768356 (+ 182us) server_negotiation.cc:299] Negotiation successful
0810 18:43:07.768432 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 18:43:07.777904 21049 raft_consensus.cc:1261] T e36f01e2c74346e0b7738f21716d2e65 P ca7f5741d77a4b1b9bf1fdc572b7d8f6 [term 1 FOLLOWER]: Refusing update from remote peer 7d0120ef53b244c6a1e0e329b7ce23be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:43:07.777909 21257 raft_consensus.cc:1261] T e36f01e2c74346e0b7738f21716d2e65 P 32a28614ccb146f29e0fb95dd7f9c8fa [term 1 FOLLOWER]: Refusing update from remote peer 7d0120ef53b244c6a1e0e329b7ce23be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:43:07.779095 21560 consensus_queue.cc:1035] T e36f01e2c74346e0b7738f21716d2e65 P 7d0120ef53b244c6a1e0e329b7ce23be [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca7f5741d77a4b1b9bf1fdc572b7d8f6" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:43:07.779489 21557 consensus_queue.cc:1035] T e36f01e2c74346e0b7738f21716d2e65 P 7d0120ef53b244c6a1e0e329b7ce23be [LEADER]: Connected to new peer: Peer: permanent_uuid: "32a28614ccb146f29e0fb95dd7f9c8fa" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:43:07.807999 20831 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:43:07.808365 20831 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:43:07.812767 20831 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6a36c9ad578b45af9c80bc9e754b7105: Sending DeleteTablet for 3 replicas of tablet e36f01e2c74346e0b7738f21716d2e65
I20260810 18:43:07.814117 21420 tablet_service.cc:1514] Processing DeleteTablet for tablet e36f01e2c74346e0b7738f21716d2e65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:43:07 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:44118
I20260810 18:43:07.814095 21017 tablet_service.cc:1514] Processing DeleteTablet for tablet e36f01e2c74346e0b7738f21716d2e65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:43:07 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:60702
I20260810 18:43:07.814482 21226 tablet_service.cc:1514] Processing DeleteTablet for tablet e36f01e2c74346e0b7738f21716d2e65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:43:07 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:48830
I20260810 18:43:07.815603 21585 tablet_replica.cc:330] T e36f01e2c74346e0b7738f21716d2e65 P ca7f5741d77a4b1b9bf1fdc572b7d8f6: stopping tablet replica
I20260810 18:43:07.816154 21585 raft_consensus.cc:2229] T e36f01e2c74346e0b7738f21716d2e65 P ca7f5741d77a4b1b9bf1fdc572b7d8f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:43:07.816752 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 20925
I20260810 18:43:07.829452 21589 tablet_replica.cc:330] T e36f01e2c74346e0b7738f21716d2e65 P 32a28614ccb146f29e0fb95dd7f9c8fa: stopping tablet replica
I20260810 18:43:07.829598 21585 raft_consensus.cc:2258] T e36f01e2c74346e0b7738f21716d2e65 P ca7f5741d77a4b1b9bf1fdc572b7d8f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:43:07.830518 21589 raft_consensus.cc:2229] T e36f01e2c74346e0b7738f21716d2e65 P 32a28614ccb146f29e0fb95dd7f9c8fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:43:07.838219 21589 raft_consensus.cc:2258] T e36f01e2c74346e0b7738f21716d2e65 P 32a28614ccb146f29e0fb95dd7f9c8fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:43:07.840709 21587 tablet_replica.cc:330] T e36f01e2c74346e0b7738f21716d2e65 P 7d0120ef53b244c6a1e0e329b7ce23be: stopping tablet replica
I20260810 18:43:07.841087 21585 ts_tablet_manager.cc:1905] T e36f01e2c74346e0b7738f21716d2e65 P ca7f5741d77a4b1b9bf1fdc572b7d8f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:43:07.842633 21589 ts_tablet_manager.cc:1905] T e36f01e2c74346e0b7738f21716d2e65 P 32a28614ccb146f29e0fb95dd7f9c8fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:43:07.848532 21585 ts_tablet_manager.cc:1918] T e36f01e2c74346e0b7738f21716d2e65 P ca7f5741d77a4b1b9bf1fdc572b7d8f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:43:07.848865 21585 log.cc:1196] T e36f01e2c74346e0b7738f21716d2e65 P ca7f5741d77a4b1b9bf1fdc572b7d8f6: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-0/wal/wals/e36f01e2c74346e0b7738f21716d2e65
I20260810 18:43:07.849562 21585 ts_tablet_manager.cc:1939] T e36f01e2c74346e0b7738f21716d2e65 P ca7f5741d77a4b1b9bf1fdc572b7d8f6: Deleting consensus metadata
I20260810 18:43:07.849565 21587 raft_consensus.cc:2229] T e36f01e2c74346e0b7738f21716d2e65 P 7d0120ef53b244c6a1e0e329b7ce23be [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:43:07.850373 21587 raft_consensus.cc:2258] T e36f01e2c74346e0b7738f21716d2e65 P 7d0120ef53b244c6a1e0e329b7ce23be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:43:07.850920 21589 ts_tablet_manager.cc:1918] T e36f01e2c74346e0b7738f21716d2e65 P 32a28614ccb146f29e0fb95dd7f9c8fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:43:07.851193 21589 log.cc:1196] T e36f01e2c74346e0b7738f21716d2e65 P 32a28614ccb146f29e0fb95dd7f9c8fa: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-1/wal/wals/e36f01e2c74346e0b7738f21716d2e65
I20260810 18:43:07.851890 21589 ts_tablet_manager.cc:1939] T e36f01e2c74346e0b7738f21716d2e65 P 32a28614ccb146f29e0fb95dd7f9c8fa: Deleting consensus metadata
I20260810 18:43:07.852128 20814 catalog_manager.cc:4841] TS ca7f5741d77a4b1b9bf1fdc572b7d8f6 (127.4.228.193:43887): tablet e36f01e2c74346e0b7738f21716d2e65 (table test-table [id=9fc50163adc84f68af950f4161ed4a83]) successfully deleted
Aug 10 18:43:07 dist-test-slave-7kzw krb5kdc[20753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.193@KRBTEST.COM
I20260810 18:43:07.854003 21587 ts_tablet_manager.cc:1905] T e36f01e2c74346e0b7738f21716d2e65 P 7d0120ef53b244c6a1e0e329b7ce23be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:43:07.854038 20812 catalog_manager.cc:4841] TS 32a28614ccb146f29e0fb95dd7f9c8fa (127.4.228.194:44293): tablet e36f01e2c74346e0b7738f21716d2e65 (table test-table [id=9fc50163adc84f68af950f4161ed4a83]) successfully deleted
I20260810 18:43:07.860174 21587 ts_tablet_manager.cc:1918] T e36f01e2c74346e0b7738f21716d2e65 P 7d0120ef53b244c6a1e0e329b7ce23be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:43:07.860400 21587 log.cc:1196] T e36f01e2c74346e0b7738f21716d2e65 P 7d0120ef53b244c6a1e0e329b7ce23be: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387267874959-5011-0/minicluster-data/ts-2/wal/wals/e36f01e2c74346e0b7738f21716d2e65
I20260810 18:43:07.860944 21587 ts_tablet_manager.cc:1939] T e36f01e2c74346e0b7738f21716d2e65 P 7d0120ef53b244c6a1e0e329b7ce23be: Deleting consensus metadata
I20260810 18:43:07.862952 20810 catalog_manager.cc:4841] TS 7d0120ef53b244c6a1e0e329b7ce23be (127.4.228.195:33547): tablet e36f01e2c74346e0b7738f21716d2e65 (table test-table [id=9fc50163adc84f68af950f4161ed4a83]) successfully deleted
I20260810 18:43:07.863621 21536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:43:07.817773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60706 (local address 127.4.228.193:43887)
0810 18:43:07.818022 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0810 18:43:07.818032 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:43:07.826149 (+ 8117us) server_negotiation.cc:407] Connection header received
0810 18:43:07.827340 (+ 1191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:43:07.827352 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:43:07.827492 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:43:07.827610 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:43:07.830590 (+ 2980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:43:07.832199 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:43:07.835105 (+ 2906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:43:07.836139 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:43:07.836289 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:43:07.854943 (+ 18654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:43:07.854963 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:43:07.854984 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:43:07.855039 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:43:07.859589 (+ 4550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:43:07.860722 (+ 1133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:43:07.860735 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:43:07.860739 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:43:07.860935 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:43:07.861556 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:43:07.861561 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:43:07.861564 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:43:07.862101 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:43:07.862448 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0810 18:43:07.863005 (+ 557us) server_negotiation.cc:299] Negotiation successful
0810 18:43:07.863256 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 18:43:07.864401 21108 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:43:07.958768 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 20925
I20260810 18:43:07.984632 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 21127
Aug 10 18:43:07 dist-test-slave-7kzw krb5kdc[20753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.194@KRBTEST.COM
I20260810 18:43:08.005620 21540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:43:07.985705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48842 (local address 127.4.228.194:44293)
0810 18:43:07.985917 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 18:43:07.985922 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:43:07.985942 (+ 20us) server_negotiation.cc:407] Connection header received
0810 18:43:07.986067 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:43:07.986071 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:43:07.986138 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:43:07.986233 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:43:07.988089 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:43:07.988861 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:43:07.991122 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:43:07.991544 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:43:07.991648 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:43:07.997683 (+ 6035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:43:07.997702 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:43:07.997722 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:43:07.997766 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:43:08.001952 (+ 4186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:43:08.002932 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:43:08.002944 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:43:08.002949 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:43:08.003139 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:43:08.003622 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:43:08.003628 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:43:08.003632 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:43:08.004212 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:43:08.004595 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0810 18:43:08.004862 (+ 267us) server_negotiation.cc:299] Negotiation successful
0810 18:43:08.005148 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260810 18:43:08.006579 21306 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:43:08.100636 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 21127
I20260810 18:43:08.134538 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 21328
Aug 10 18:43:08 dist-test-slave-7kzw krb5kdc[20753](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.195@KRBTEST.COM
I20260810 18:43:08.156672 21538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:43:08.135374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44148 (local address 127.4.228.195:33547)
0810 18:43:08.135577 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 18:43:08.135582 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:43:08.135839 (+ 257us) server_negotiation.cc:407] Connection header received
0810 18:43:08.136020 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:43:08.136025 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:43:08.136140 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:43:08.136242 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:43:08.138279 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:43:08.139284 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:43:08.141337 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:43:08.141897 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:43:08.142022 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:43:08.148853 (+ 6831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:43:08.148870 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:43:08.148889 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:43:08.148936 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:43:08.152671 (+ 3735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:43:08.153769 (+ 1098us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:43:08.153778 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:43:08.153780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:43:08.153952 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:43:08.154460 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:43:08.154527 (+ 67us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:43:08.154534 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:43:08.155018 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:43:08.155284 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0810 18:43:08.155778 (+ 494us) server_negotiation.cc:299] Negotiation successful
0810 18:43:08.156094 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 18:43:08.157550 21505 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:43:08.288210 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 21328
I20260810 18:43:08.314699 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 20780
I20260810 18:43:08.316231 20879 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:43:08.431807 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 20780
2026-08-10T18:43:08Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (5185 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:43:08 dist-test-slave-7kzw krb5kdc[21601](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:43:08 dist-test-slave-7kzw krb5kdc[21601](info): set up 2 sockets
Aug 10 18:43:08 dist-test-slave-7kzw krb5kdc[21601](info): commencing operation
krb5kdc: starting...
W20260810 18:43:09.836838 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.350s user 0.001s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:43:10 dist-test-slave-7kzw krb5kdc[21601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:43:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:43:10Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.4.228.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.4.228.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.4.228.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/rangeradmin_127.4.228.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.4.228.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/rangeradmin_127.4.228.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.4.228.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.4.228.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.4.228.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/rangerlookup_127.4.228.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.4.228.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/rangerlookup_127.4.228.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.228.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.4.228.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.228.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/HTTP_127.4.228.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.228.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/HTTP_127.4.228.212@KRBTEST.COM.keytab.
I20260810 18:43:10.324235 5011 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskTAuB_V/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/postgres -l logfile start
2026-08-10 18:43:13.308 UTC [21646] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 18:43:13.309 UTC [21646] LOG: listening on IPv4 address "127.4.228.212", port 57611
2026-08-10 18:43:13.313 UTC [21646] LOG: listening on Unix socket "/tmp/.s.PGSQL.57611"
2026-08-10 18:43:13.338 UTC [21649] LOG: database system was shut down at 2026-08-10 18:43:11 UTC
2026-08-10 18:43:13.342 UTC [21646] LOG: database system is ready to accept connections
I20260810 18:43:14.597980 5011 mini_postgres.cc:96] Postgres bound to 57611
2026-08-10 18:43:14.626 UTC [21661] FATAL: database "slave" does not exist
127.4.228.212:57611 - accepting connections
I20260810 18:43:14.627220 5011 mini_ranger.cc:161] Starting Ranger...
I20260810 18:43:14.666985 5011 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 18:43:15.311514 5011 mini_ranger.cc:87] Created ranger Postgres database
I20260810 18:43:15.311697 5011 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ranger-admin
2026-08-10 18:43:15,659 [I] DB FLAVOR :POSTGRES
2026-08-10 18:43:15,659 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 18:43:15,659 [I] Checking connection..
2026-08-10 18:43:15,660 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:57611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 18:43:15,953 [I] Checking connection passed.
2026-08-10 18:43:15,953 [I] --------- Verifying version history table ---------
2026-08-10 18:43:15,954 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:57611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 18:43:16,242 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 18:43:16,243 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:57611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 18:43:16,516 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 18:43:16,516 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 18:43:16,516 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:57611/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 18:43:16.795 UTC [21763] WARNING: there is no transaction in progress
2026-08-10 18:43:16,805 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 18:43:16,806 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:57611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 18:43:17,090 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 18:43:17,091 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 18:43:17,091 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:57611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 18:43:17,380 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:57611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 18:43:17,656 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:57611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-7kzw.c.gcp-upstream.internal','N') ;"
2026-08-10 18:43:17,948 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:57611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 18:43:18,236 [I] Table x_portal_user does not exist in database ranger
2026-08-10 18:43:18,237 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:57611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 18:43:18,518 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 18:43:18,519 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:57611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 18:43:18,788 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:57611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 18:43:19,060 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 18:43:19,061 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:57611/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 18:43:20.172 UTC [21936] WARNING: there is no transaction in progress
2026-08-10 18:43:20.193 UTC [21936] WARNING: there is no transaction in progress
2026-08-10 18:43:20.214 UTC [21936] WARNING: there is no transaction in progress
2026-08-10 18:43:20.235 UTC [21936] WARNING: there is no transaction in progress
2026-08-10 18:43:20.259 UTC [21936] WARNING: there is no transaction in progress
2026-08-10 18:43:20.281 UTC [21936] WARNING: there is no transaction in progress
2026-08-10 18:43:20.352 UTC [21936] WARNING: there is no transaction in progress
2026-08-10 18:43:20.361 UTC [21936] WARNING: there is no transaction in progress
2026-08-10 18:43:20.378 UTC [21936] WARNING: there is no transaction in progress
2026-08-10 18:43:20.391 UTC [21936] WARNING: there is no transaction in progress
2026-08-10 18:43:20.403 UTC [21936] WARNING: there is no transaction in progress
2026-08-10 18:43:20.421 UTC [21936] WARNING: there is no transaction in progress
2026-08-10 18:43:20.432 UTC [21936] WARNING: there is no transaction in progress
2026-08-10 18:43:20.439 UTC [21936] WARNING: there is no transaction in progress
2026-08-10 18:43:20.448 UTC [21936] WARNING: there is no transaction in progress
2026-08-10 18:43:20.457 UTC [21936] WARNING: there is no transaction in progress
2026-08-10 18:43:21,339 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 18:43:21,339 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:57611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 18:43:21,590 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 18:43:21,591 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:57611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 18:43:21,861 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 18:43:21,861 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:57611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 18:43:22,157 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 18:43:22,158 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:57611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 18:43:22,462 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:57611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 18:43:22,732 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:57611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-7kzw.c.gcp-upstream.internal';"
2026-08-10 18:43:23,012 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 18:43:23,013 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:57611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 18:43:23,273 [I] DB_PATCHES have already been applied
I20260810 18:43:23.280601 5011 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ranger-admin:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskTAuB_V/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 18:43:23.280736 5011 mini_ranger.cc:193] Using host: 127.4.228.212
I20260810 18:43:23.307530 5011 mini_ranger.cc:227] Ranger admin URL: http://127.4.228.212:47401
Aug 10, 2026 6:43:23 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 6:43:23 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 6:43:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 6:43:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 6:43:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 6:43:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 6:43:24 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 18:43:24.848724 5011 mini_ranger.cc:160] Time spent starting Ranger: real 10.221s user 0.002s sys 0.075s
Aug 10, 2026 6:43:47 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 6:43:47 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 10, 2026 6:43:47 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 6:43:47 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 6:43:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 6:43:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 6:43:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 6:43:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 6:43:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 6:43:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 6:43:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 6:43:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 6:43:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 6:43:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 6:43:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 6:43:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 6:43:47 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 6:43:48 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 18:43:48.636901 5011 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.4.228.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.4.228.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.4.228.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/rangerkms_127.4.228.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.4.228.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/rangerkms_127.4.228.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.228.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.4.228.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.228.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/HTTP_127.4.228.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.228.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/HTTP_127.4.228.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 18:43:48 dist-test-slave-7kzw krb5kdc[21601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387428, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 18:43:48.883188 5011 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 18:43:48.914784 5011 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 18:43:50.015988 5011 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 18:43:50.016175 5011 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ranger-kms
I20260810 18:43:50.016360 5011 mini_ranger_kms.cc:227] Using postgres at 127.4.228.212:57611
2026-08-10 18:43:50,934 [I] DB FLAVOR :POSTGRES
2026-08-10 18:43:50,935 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 18:43:50,935 [I] Checking connection
2026-08-10 18:43:50,935 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:57611/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 18:43:51,214 [I] connection success
2026-08-10 18:43:51,215 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 18:43:51,215 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 18:43:51,215 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:57611/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 18:43:51,489 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 18:43:51,489 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 18:43:51,490 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 18:43:51,490 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:57611/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 18:43:51,789 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 18:43:52.326984 5011 mini_ranger_kms.cc:310] Created kms service
I20260810 18:43:52.524317 5011 mini_ranger_kms.cc:326] Created kudu user
I20260810 18:43:52.574836 5011 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 18:43:52 dist-test-slave-7kzw krb5kdc[21601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387428, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.4.228.212@KRBTEST.COM
I20260810 18:43:53.021792 5011 mini_ranger_kms.cc:384] Added ranger policy
I20260810 18:43:53.023401 5011 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ranger-kms:/tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 18:43:53.023514 5011 mini_ranger_kms.cc:240] Using host: 127.4.228.212
I20260810 18:43:53.051052 5011 mini_ranger_kms.cc:276] Ranger KMS PID: 22223
I20260810 18:43:53.051280 5011 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.4.228.212:57683
Aug 10, 2026 6:43:54 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 6:43:54 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 6:43:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 6:43:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 6:43:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 6:43:54 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 18:43:55 dist-test-slave-7kzw krb5kdc[21601](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786387435, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.228.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:43:55.996990 5011 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 7.114s user 0.170s sys 0.503s
I20260810 18:43:55.997370 5011 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 18:43:55.997607 5011 mini_ranger_kms.cc:401] 127.4.228.212:57683/kms/v1/keys
Aug 10 18:43:56 dist-test-slave-7kzw krb5kdc[21601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387435, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.228.212@KRBTEST.COM for HTTP/127.4.228.212@KRBTEST.COM
Aug 10 18:43:56 dist-test-slave-7kzw krb5kdc[21601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387435, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.228.212@KRBTEST.COM for HTTP/127.4.228.212@KRBTEST.COM
Aug 10 18:43:59 dist-test-slave-7kzw krb5kdc[21601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.4.228.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
I20260810 18:43:59.415351 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:40945
--webserver_interface=127.4.228.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:42733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:40945
--ranger_config_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.228.212:57683/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:44:00.100112 22298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:44:00.100672 22298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:44:00.100929 22298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:44:00.112326 22298 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:44:00.112648 22298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:44:00.112818 22298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:44:00.113006 22298 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:44:00.113168 22298 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:44:00.128430 22298 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:42733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/master-0/wal
--ranger_kms_url=127.4.228.212:57683/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:40945
--ranger_config_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:40945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:44:00.130223 22298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:44:00.131981 22298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:44:00.151716 22311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:44:00.151793 22320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:44:00.152136 22314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:44:00.154173 22298 server_base.cc:1034] running on GCE node
I20260810 18:44:00.155272 22298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:44:00.157119 22298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:44:00.158407 22298 hybrid_clock.cc:648] HybridClock initialized: now 1786387440158299 us; error 64 us; skew 500 ppm
I20260810 18:44:00.159017 22298 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:44:00 dist-test-slave-7kzw krb5kdc[21601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387440, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:44:00.169585 22298 init.cc:377] Logged in from keytab as kudu/127.4.228.254@KRBTEST.COM (short username kudu)
I20260810 18:44:00.172482 22298 webserver.cc:460] Webserver started at http://127.4.228.254:34287/ using document root <none> and password file <none>
I20260810 18:44:00.173561 22298 fs_manager.cc:359] Metadata directory not provided
I20260810 18:44:00.173846 22298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:44:00.174387 22298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 18:44:00 dist-test-slave-7kzw krb5kdc[21601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387440, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for HTTP/127.4.228.212@KRBTEST.COM
I20260810 18:44:00.250809 22298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "4b6da28a44ba42aeb88c59fb8b7401a4"
format_stamp: "Formatted at 2026-08-10 18:44:00 on dist-test-slave-7kzw"
server_key: "e419614507ce12f23e0672bc6e766455"
server_key_iv: "c424a8fdafaea6e68b8872c244bd176d"
server_key_version: "kuduclusterkey@0"
I20260810 18:44:00.252216 22298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "4b6da28a44ba42aeb88c59fb8b7401a4"
format_stamp: "Formatted at 2026-08-10 18:44:00 on dist-test-slave-7kzw"
server_key: "e419614507ce12f23e0672bc6e766455"
server_key_iv: "c424a8fdafaea6e68b8872c244bd176d"
server_key_version: "kuduclusterkey@0"
I20260810 18:44:00.261168 22298 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.004s sys 0.004s
I20260810 18:44:00.284209 22332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:44:00.286531 22298 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.002s
I20260810 18:44:00.286862 22298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "4b6da28a44ba42aeb88c59fb8b7401a4"
format_stamp: "Formatted at 2026-08-10 18:44:00 on dist-test-slave-7kzw"
server_key: "e419614507ce12f23e0672bc6e766455"
server_key_iv: "c424a8fdafaea6e68b8872c244bd176d"
server_key_version: "kuduclusterkey@0"
I20260810 18:44:00.287213 22298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:44:00.319825 22298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:44:00.327486 22298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:44:00.328109 22298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:44:00.373729 22298 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:40945
I20260810 18:44:00.373793 22417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:40945 every 8 connection(s)
I20260810 18:44:00.376232 22298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:44:00.378841 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 22298
I20260810 18:44:00.379237 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:44:00.380460 5011 external_mini_cluster.cc:1428] Setting key ce334b6f2de438d8142c5896445c4e7f
I20260810 18:44:00.383953 22418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 18:44:00 dist-test-slave-7kzw krb5kdc[21601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:44:00.408491 22418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b6da28a44ba42aeb88c59fb8b7401a4: Bootstrap starting.
I20260810 18:44:00.415717 22418 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b6da28a44ba42aeb88c59fb8b7401a4: Neither blocks nor log segments found. Creating new log.
I20260810 18:44:00.418321 22418 log.cc:824] T 00000000000000000000000000000000 P 4b6da28a44ba42aeb88c59fb8b7401a4: Log is configured to *not* fsync() on all Append() calls
I20260810 18:44:00.418674 22425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:44:00.383679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41554 (local address 127.4.228.254:40945)
0810 18:44:00.385423 (+ 1744us) server_negotiation.cc:206] Beginning negotiation
0810 18:44:00.385441 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 18:44:00.385530 (+ 89us) server_negotiation.cc:407] Connection header received
0810 18:44:00.389172 (+ 3642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:44:00.389210 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:44:00.389973 (+ 763us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:44:00.390972 (+ 999us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:44:00.392284 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:00.395937 (+ 3653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:00.397475 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:00.399011 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:00.399279 (+ 268us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:44:00.405075 (+ 5796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:44:00.405091 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:44:00.405118 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:44:00.405221 (+ 103us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:44:00.411633 (+ 6412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:44:00.412714 (+ 1081us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:44:00.412727 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:44:00.412747 (+ 20us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:44:00.413072 (+ 325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:44:00.413672 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:44:00.413677 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:44:00.413680 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:44:00.414789 (+ 1109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:44:00.415269 (+ 480us) server_negotiation.cc:1035] Waiting for connection context
0810 18:44:00.415721 (+ 452us) server_negotiation.cc:299] Negotiation successful
0810 18:44:00.416273 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1407,"thread_start_us":967,"threads_started":1}
I20260810 18:44:00.424448 22418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b6da28a44ba42aeb88c59fb8b7401a4: No bootstrap required, opened a new log
I20260810 18:44:00.435998 22418 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b6da28a44ba42aeb88c59fb8b7401a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b6da28a44ba42aeb88c59fb8b7401a4" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 40945 } }
I20260810 18:44:00.436722 22418 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b6da28a44ba42aeb88c59fb8b7401a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:44:00.436866 22418 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b6da28a44ba42aeb88c59fb8b7401a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b6da28a44ba42aeb88c59fb8b7401a4, State: Initialized, Role: FOLLOWER
I20260810 18:44:00.437623 22418 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b6da28a44ba42aeb88c59fb8b7401a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b6da28a44ba42aeb88c59fb8b7401a4" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 40945 } }
I20260810 18:44:00.438104 22418 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b6da28a44ba42aeb88c59fb8b7401a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:44:00.438412 22418 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b6da28a44ba42aeb88c59fb8b7401a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:44:00.438818 22418 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b6da28a44ba42aeb88c59fb8b7401a4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:44:00.442054 22418 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b6da28a44ba42aeb88c59fb8b7401a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b6da28a44ba42aeb88c59fb8b7401a4" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 40945 } }
I20260810 18:44:00.442898 22418 leader_election.cc:302] T 00000000000000000000000000000000 P 4b6da28a44ba42aeb88c59fb8b7401a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b6da28a44ba42aeb88c59fb8b7401a4; no voters:
I20260810 18:44:00.444119 22418 leader_election.cc:288] T 00000000000000000000000000000000 P 4b6da28a44ba42aeb88c59fb8b7401a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:44:00.444295 22430 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b6da28a44ba42aeb88c59fb8b7401a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:44:00.445945 22430 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b6da28a44ba42aeb88c59fb8b7401a4 [term 1 LEADER]: Becoming Leader. State: Replica: 4b6da28a44ba42aeb88c59fb8b7401a4, State: Running, Role: LEADER
I20260810 18:44:00.446906 22430 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b6da28a44ba42aeb88c59fb8b7401a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b6da28a44ba42aeb88c59fb8b7401a4" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 40945 } }
I20260810 18:44:00.447856 22418 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b6da28a44ba42aeb88c59fb8b7401a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:44:00.452184 22432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b6da28a44ba42aeb88c59fb8b7401a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b6da28a44ba42aeb88c59fb8b7401a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b6da28a44ba42aeb88c59fb8b7401a4" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 40945 } } }
I20260810 18:44:00.451419 22433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b6da28a44ba42aeb88c59fb8b7401a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b6da28a44ba42aeb88c59fb8b7401a4. Latest consensus state: current_term: 1 leader_uuid: "4b6da28a44ba42aeb88c59fb8b7401a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b6da28a44ba42aeb88c59fb8b7401a4" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 40945 } } }
I20260810 18:44:00.453198 22433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b6da28a44ba42aeb88c59fb8b7401a4 [sys.catalog]: This master's current role is: LEADER
I20260810 18:44:00.453205 22432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b6da28a44ba42aeb88c59fb8b7401a4 [sys.catalog]: This master's current role is: LEADER
I20260810 18:44:00.456820 22437 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:44:00.468351 22437 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:44:00.469810 22418 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 18:44:00.495491 22437 catalog_manager.cc:1347] Generated new cluster ID: 5ff4600a74314fc093d8e561e4a3833b
I20260810 18:44:00.495759 22437 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:44:00.513007 22437 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:44:00.514767 22437 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:44:00.522984 22437 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b6da28a44ba42aeb88c59fb8b7401a4: Generated new TSK 0
I20260810 18:44:00.524181 22437 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.228.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab.
Entry for principal kudu/127.4.228.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab.
I20260810 18:44:01.502946 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.228.212:57683/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.193:0
--local_ip_for_outbound_sockets=127.4.228.193
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:40945
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:42733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:44:01 dist-test-slave-7kzw krb5kdc[21601](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786387441, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 18:44:02.286028 22490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:44:02.286742 22490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:44:02.287113 22490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:44:02.305783 22490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:44:02.306111 22490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:44:02.306701 22490 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.193
I20260810 18:44:02.330957 22490 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:42733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.4.228.212:57683/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:40945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.193
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:44:02.333895 22490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:44:02.337047 22490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:44:02.363673 22508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:44:02.365592 22509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:44:02.383991 22490 server_base.cc:1034] running on GCE node
W20260810 18:44:02.385749 22514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:44:02.388266 22490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:44:02.391939 22490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:44:02.395586 22490 hybrid_clock.cc:648] HybridClock initialized: now 1786387442395400 us; error 174 us; skew 500 ppm
I20260810 18:44:02.396286 22490 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:44:02 dist-test-slave-7kzw krb5kdc[21601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387442, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:44:02.403829 22490 init.cc:377] Logged in from keytab as kudu/127.4.228.193@KRBTEST.COM (short username kudu)
I20260810 18:44:02.422104 22490 webserver.cc:460] Webserver started at http://127.4.228.193:40059/ using document root <none> and password file <none>
I20260810 18:44:02.423652 22490 fs_manager.cc:359] Metadata directory not provided
I20260810 18:44:02.423918 22490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:44:02.424659 22490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 18:44:02 dist-test-slave-7kzw krb5kdc[21601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387442, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for HTTP/127.4.228.212@KRBTEST.COM
Aug 10 18:44:02 dist-test-slave-7kzw krb5kdc[21601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387441, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for HTTP/127.4.228.212@KRBTEST.COM
I20260810 18:44:02.455024 22490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-0/data/instance:
uuid: "f6565bf8d91346f090f3c3906ebda532"
format_stamp: "Formatted at 2026-08-10 18:44:02 on dist-test-slave-7kzw"
server_key: "2eba380c264d102f9ce7b7ee85a3a24e"
server_key_iv: "fcadb90d71e9eaa005f66f0e7737a7ed"
server_key_version: "kuduclusterkey@0"
I20260810 18:44:02.456526 22490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance:
uuid: "f6565bf8d91346f090f3c3906ebda532"
format_stamp: "Formatted at 2026-08-10 18:44:02 on dist-test-slave-7kzw"
server_key: "2eba380c264d102f9ce7b7ee85a3a24e"
server_key_iv: "fcadb90d71e9eaa005f66f0e7737a7ed"
server_key_version: "kuduclusterkey@0"
I20260810 18:44:02.488338 22490 fs_manager.cc:691] Time spent creating directory manager: real 0.031s user 0.020s sys 0.008s
I20260810 18:44:02.530927 22521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:44:02.544371 22490 fs_manager.cc:725] Time spent opening block manager: real 0.029s user 0.002s sys 0.002s
I20260810 18:44:02.544760 22490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-0/wal
uuid: "f6565bf8d91346f090f3c3906ebda532"
format_stamp: "Formatted at 2026-08-10 18:44:02 on dist-test-slave-7kzw"
server_key: "2eba380c264d102f9ce7b7ee85a3a24e"
server_key_iv: "fcadb90d71e9eaa005f66f0e7737a7ed"
server_key_version: "kuduclusterkey@0"
I20260810 18:44:02.545343 22490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:44:02.614708 22490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:44:02.621354 22490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:44:02.621937 22490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:44:02.623751 22490 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:44:02.626885 22490 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:44:02.627022 22490 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:44:02.627261 22490 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:44:02.627403 22490 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:44:02.773689 22490 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.193:41075
I20260810 18:44:02.773682 22696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.193:41075 every 8 connection(s)
I20260810 18:44:02.776304 22490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
I20260810 18:44:02.785254 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 22490
I20260810 18:44:02.785522 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance
I20260810 18:44:02.786646 5011 external_mini_cluster.cc:1428] Setting key 049012260c673a05b6cd9dc4af898864
WARNING: no policy specified for kudu/127.4.228.194@KRBTEST.COM; defaulting to no policy
Aug 10 18:44:02 dist-test-slave-7kzw krb5kdc[21601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387442, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.194@KRBTEST.COM" created.
I20260810 18:44:02.879763 22700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:44:02.781778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:41191 (local address 127.4.228.254:40945)
0810 18:44:02.818819 (+ 37041us) server_negotiation.cc:206] Beginning negotiation
0810 18:44:02.818827 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:44:02.822594 (+ 3767us) server_negotiation.cc:407] Connection header received
0810 18:44:02.833775 (+ 11181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:44:02.833782 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:44:02.833887 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:44:02.834007 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:44:02.845579 (+ 11572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:02.846848 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:02.851315 (+ 4467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:02.852161 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:02.852316 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:44:02.861382 (+ 9066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:44:02.861396 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:44:02.861401 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:44:02.861460 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:44:02.868759 (+ 7299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:44:02.875623 (+ 6864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:44:02.875662 (+ 39us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:44:02.875669 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:44:02.876017 (+ 348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:44:02.876986 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:44:02.876992 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:44:02.876996 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:44:02.877458 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:44:02.877644 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0810 18:44:02.878957 (+ 1313us) server_negotiation.cc:299] Negotiation successful
0810 18:44:02.879255 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36865,"thread_start_us":36707,"threads_started":1}
I20260810 18:44:02.881070 22699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:44:02.780969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:40945 (local address 127.4.228.193:41191)
0810 18:44:02.819631 (+ 38662us) negotiation.cc:107] Waiting for socket to connect
0810 18:44:02.819737 (+ 106us) client_negotiation.cc:174] Beginning negotiation
0810 18:44:02.833302 (+ 13565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:44:02.836984 (+ 3682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:44:02.837003 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:44:02.843785 (+ 6782us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:44:02.845211 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:44:02.845261 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:44:02.847101 (+ 1840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:44:02.847110 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:44:02.850950 (+ 3840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:44:02.850989 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:44:02.852563 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:44:02.852585 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:44:02.852998 (+ 413us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:44:02.854060 (+ 1062us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:44:02.854132 (+ 72us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:44:02.860850 (+ 6718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:44:02.871687 (+ 10837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:44:02.871703 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:44:02.871756 (+ 53us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:44:02.872628 (+ 872us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:44:02.876355 (+ 3727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:44:02.876362 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:44:02.876365 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:44:02.876763 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:44:02.877759 (+ 996us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:44:02.877771 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:44:02.878339 (+ 568us) client_negotiation.cc:769] Sending connection context
0810 18:44:02.878781 (+ 442us) client_negotiation.cc:240] Negotiation successful
0810 18:44:02.879431 (+ 650us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38217,"thread_start_us":49406,"threads_started":1}
Aug 10 18:44:02 dist-test-slave-7kzw krb5kdc[21601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387441, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for HTTP/127.4.228.212@KRBTEST.COM
I20260810 18:44:02.883464 22697 heartbeater.cc:347] Connected to a master server at 127.4.228.254:40945
I20260810 18:44:02.883852 22697 heartbeater.cc:464] Registering TS with master...
I20260810 18:44:02.884881 22697 heartbeater.cc:511] Master 127.4.228.254:40945 requested a full tablet report, sending...
I20260810 18:44:02.888322 22355 ts_manager.cc:194] Registered new tserver with Master: f6565bf8d91346f090f3c3906ebda532 (127.4.228.193:41075)
I20260810 18:44:02.891291 22355 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.193@KRBTEST.COM'} at 127.4.228.193:41191
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab.
Entry for principal kudu/127.4.228.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab.
W20260810 18:44:02.925469 5011 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.4.228.194: real 0.135s user 0.000s sys 0.027s
I20260810 18:44:02.927089 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.228.212:57683/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.194:0
--local_ip_for_outbound_sockets=127.4.228.194
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:40945
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:42733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:44:03.606397 22711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:44:03.606875 22711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:44:03.607120 22711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:44:03.617372 22711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:44:03.617664 22711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:44:03.617964 22711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.194
I20260810 18:44:03.632453 22711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:42733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.4.228.212:57683/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:40945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.194
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:44:03.634078 22711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:44:03.635835 22711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:44:03.652966 22728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:44:03.653110 22730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:44:03.654135 22733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:44:03.656028 22711 server_base.cc:1034] running on GCE node
I20260810 18:44:03.656711 22711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:44:03.657743 22711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:44:03.658954 22711 hybrid_clock.cc:648] HybridClock initialized: now 1786387443658901 us; error 34 us; skew 500 ppm
I20260810 18:44:03.659315 22711 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:44:03 dist-test-slave-7kzw krb5kdc[21601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387443, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:44:03.665300 22711 init.cc:377] Logged in from keytab as kudu/127.4.228.194@KRBTEST.COM (short username kudu)
I20260810 18:44:03.668578 22711 webserver.cc:460] Webserver started at http://127.4.228.194:42659/ using document root <none> and password file <none>
I20260810 18:44:03.669579 22711 fs_manager.cc:359] Metadata directory not provided
I20260810 18:44:03.669785 22711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:44:03.670264 22711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 18:44:03 dist-test-slave-7kzw krb5kdc[21601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387443, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for HTTP/127.4.228.212@KRBTEST.COM
I20260810 18:44:03.690593 22711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-1/data/instance:
uuid: "1c2367a7718142f1bbcd46143de6e9a0"
format_stamp: "Formatted at 2026-08-10 18:44:03 on dist-test-slave-7kzw"
server_key: "fdd961eb5d5505540b6da73faff54ab6"
server_key_iv: "14722a870ea4f5ad997bb377805c2a11"
server_key_version: "kuduclusterkey@0"
I20260810 18:44:03.691946 22711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance:
uuid: "1c2367a7718142f1bbcd46143de6e9a0"
format_stamp: "Formatted at 2026-08-10 18:44:03 on dist-test-slave-7kzw"
server_key: "fdd961eb5d5505540b6da73faff54ab6"
server_key_iv: "14722a870ea4f5ad997bb377805c2a11"
server_key_version: "kuduclusterkey@0"
I20260810 18:44:03.700026 22711 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.003s
I20260810 18:44:03.721047 22747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:44:03.723642 22711 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.001s
I20260810 18:44:03.723944 22711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-1/wal
uuid: "1c2367a7718142f1bbcd46143de6e9a0"
format_stamp: "Formatted at 2026-08-10 18:44:03 on dist-test-slave-7kzw"
server_key: "fdd961eb5d5505540b6da73faff54ab6"
server_key_iv: "14722a870ea4f5ad997bb377805c2a11"
server_key_version: "kuduclusterkey@0"
I20260810 18:44:03.724332 22711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:44:03.769639 22711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:44:03.776770 22711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:44:03.777488 22711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:44:03.779490 22711 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:44:03.782255 22711 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:44:03.782385 22711 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:44:03.782640 22711 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:44:03.782752 22711 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260810 18:44:03.860899 22711 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.194:41459
I20260810 18:44:03.860965 22926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.194:41459 every 8 connection(s)
I20260810 18:44:03.862540 22711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
I20260810 18:44:03.866345 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 22711
I20260810 18:44:03.866600 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance
I20260810 18:44:03.867287 5011 external_mini_cluster.cc:1428] Setting key d7f34bc1777f2f7e21478d1585df609c
Aug 10 18:44:03 dist-test-slave-7kzw krb5kdc[21601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387443, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:44:03.888381 22935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:44:03.865584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:59915 (local address 127.4.228.254:40945)
0810 18:44:03.866193 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0810 18:44:03.866198 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:44:03.867359 (+ 1161us) server_negotiation.cc:407] Connection header received
0810 18:44:03.869763 (+ 2404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:44:03.869770 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:44:03.869870 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:44:03.870008 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:44:03.871958 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:03.872825 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:03.875034 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:03.875503 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:03.875663 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:44:03.880703 (+ 5040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:44:03.880709 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:44:03.880713 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:44:03.880771 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:44:03.884064 (+ 3293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:44:03.885007 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:44:03.885012 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:44:03.885015 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:44:03.885180 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:44:03.885794 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:44:03.885802 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:44:03.885809 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:44:03.886257 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:44:03.886445 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 18:44:03.887574 (+ 1129us) server_negotiation.cc:299] Negotiation successful
0810 18:44:03.887867 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":349,"threads_started":1}
I20260810 18:44:03.889344 22936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:44:03.866074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:40945 (local address 127.4.228.194:59915)
0810 18:44:03.867156 (+ 1082us) negotiation.cc:107] Waiting for socket to connect
0810 18:44:03.867211 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 18:44:03.869404 (+ 2193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:44:03.870308 (+ 904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:44:03.870317 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:44:03.870919 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:44:03.871758 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:44:03.871780 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:44:03.873050 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:44:03.873054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:44:03.874845 (+ 1791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:44:03.874869 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:44:03.875797 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:44:03.875809 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:44:03.876214 (+ 405us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:44:03.877447 (+ 1233us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:44:03.877484 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:44:03.880411 (+ 2927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:44:03.884271 (+ 3860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:44:03.884283 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:44:03.884306 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:44:03.884860 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:44:03.885316 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:44:03.885322 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:44:03.885326 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:44:03.885575 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:44:03.886447 (+ 872us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:44:03.886455 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:44:03.887074 (+ 619us) client_negotiation.cc:769] Sending connection context
0810 18:44:03.887384 (+ 310us) client_negotiation.cc:240] Negotiation successful
0810 18:44:03.887911 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":755,"thread_start_us":497,"threads_started":1}
I20260810 18:44:03.891327 22930 heartbeater.cc:347] Connected to a master server at 127.4.228.254:40945
I20260810 18:44:03.891736 22930 heartbeater.cc:464] Registering TS with master...
I20260810 18:44:03.892670 22930 heartbeater.cc:511] Master 127.4.228.254:40945 requested a full tablet report, sending...
I20260810 18:44:03.894675 22355 ts_manager.cc:194] Registered new tserver with Master: 1c2367a7718142f1bbcd46143de6e9a0 (127.4.228.194:41459)
I20260810 18:44:03.895627 22355 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.194@KRBTEST.COM'} at 127.4.228.194:59915
WARNING: no policy specified for kudu/127.4.228.195@KRBTEST.COM; defaulting to no policy
I20260810 18:44:03.898455 22697 heartbeater.cc:503] Master 127.4.228.254:40945 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab.
Entry for principal kudu/127.4.228.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab.
I20260810 18:44:03.939903 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.228.212:57683/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.195:0
--local_ip_for_outbound_sockets=127.4.228.195
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:40945
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:42733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:44:04.527652 22940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:44:04.528098 22940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:44:04.528375 22940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:44:04.539165 22940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:44:04.539382 22940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:44:04.539676 22940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.195
I20260810 18:44:04.553085 22940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:42733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.4.228.212:57683/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:40945
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.195
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:44:04.554975 22940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:44:04.556631 22940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:44:04.577165 22949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:44:04.577451 22953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:44:04.577479 22955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:44:04.579735 22940 server_base.cc:1034] running on GCE node
I20260810 18:44:04.580492 22940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:44:04.581670 22940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:44:04.582968 22940 hybrid_clock.cc:648] HybridClock initialized: now 1786387444582865 us; error 54 us; skew 500 ppm
I20260810 18:44:04.583421 22940 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:44:04 dist-test-slave-7kzw krb5kdc[21601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387444, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:44:04.588281 22940 init.cc:377] Logged in from keytab as kudu/127.4.228.195@KRBTEST.COM (short username kudu)
I20260810 18:44:04.590528 22940 webserver.cc:460] Webserver started at http://127.4.228.195:37225/ using document root <none> and password file <none>
I20260810 18:44:04.591470 22940 fs_manager.cc:359] Metadata directory not provided
I20260810 18:44:04.591646 22940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:44:04.592029 22940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 18:44:04 dist-test-slave-7kzw krb5kdc[21601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387444, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for HTTP/127.4.228.212@KRBTEST.COM
I20260810 18:44:04.612355 22940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-2/data/instance:
uuid: "e4231229ed9d4a7fb4d455693f16b399"
format_stamp: "Formatted at 2026-08-10 18:44:04 on dist-test-slave-7kzw"
server_key: "5cc0f8285a9b2f7f7eee3d5e57bfd04a"
server_key_iv: "99eeb4b5efe82204ba6b188f30f7cbeb"
server_key_version: "kuduclusterkey@0"
I20260810 18:44:04.613967 22940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance:
uuid: "e4231229ed9d4a7fb4d455693f16b399"
format_stamp: "Formatted at 2026-08-10 18:44:04 on dist-test-slave-7kzw"
server_key: "5cc0f8285a9b2f7f7eee3d5e57bfd04a"
server_key_iv: "99eeb4b5efe82204ba6b188f30f7cbeb"
server_key_version: "kuduclusterkey@0"
I20260810 18:44:04.623265 22940 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.002s sys 0.006s
I20260810 18:44:04.642213 22968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:44:04.644023 22940 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260810 18:44:04.644383 22940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-2/wal
uuid: "e4231229ed9d4a7fb4d455693f16b399"
format_stamp: "Formatted at 2026-08-10 18:44:04 on dist-test-slave-7kzw"
server_key: "5cc0f8285a9b2f7f7eee3d5e57bfd04a"
server_key_iv: "99eeb4b5efe82204ba6b188f30f7cbeb"
server_key_version: "kuduclusterkey@0"
I20260810 18:44:04.644810 22940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:44:04.674400 22940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:44:04.680517 22940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:44:04.681151 22940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:44:04.682982 22940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:44:04.685195 22940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:44:04.685367 22940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:44:04.685606 22940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:44:04.685736 22940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:44:04.764431 22940 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.195:43777
I20260810 18:44:04.764465 23155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.195:43777 every 8 connection(s)
I20260810 18:44:04.766098 22940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
I20260810 18:44:04.770678 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 22940
I20260810 18:44:04.770890 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance
I20260810 18:44:04.771552 5011 external_mini_cluster.cc:1428] Setting key 76ead20270b1055554c417747d95fa60
Aug 10 18:44:04 dist-test-slave-7kzw krb5kdc[21601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387444, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:44:04.790788 23161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:44:04.769416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:45907 (local address 127.4.228.254:40945)
0810 18:44:04.769935 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0810 18:44:04.769940 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:44:04.772258 (+ 2318us) server_negotiation.cc:407] Connection header received
0810 18:44:04.773963 (+ 1705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:44:04.773970 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:44:04.774091 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:44:04.774226 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:44:04.776100 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:04.777263 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:04.779005 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:04.779532 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:04.779631 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:44:04.783480 (+ 3849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:44:04.783485 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:44:04.783488 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:44:04.783526 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:44:04.787196 (+ 3670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:44:04.788280 (+ 1084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:44:04.788284 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:44:04.788287 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:44:04.788435 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:44:04.788966 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:44:04.788971 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:44:04.788973 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:44:04.789299 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:44:04.789442 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 18:44:04.790187 (+ 745us) server_negotiation.cc:299] Negotiation successful
0810 18:44:04.790389 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":307,"threads_started":1}
I20260810 18:44:04.791905 23162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:44:04.770046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:40945 (local address 127.4.228.195:45907)
0810 18:44:04.771194 (+ 1148us) negotiation.cc:107] Waiting for socket to connect
0810 18:44:04.771259 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0810 18:44:04.773658 (+ 2399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:44:04.774498 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:44:04.774510 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:44:04.775168 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:44:04.775932 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:44:04.775950 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:44:04.777414 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:44:04.777418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:44:04.778834 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:44:04.778848 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:44:04.779781 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:44:04.779787 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:44:04.779984 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:44:04.780768 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:44:04.780792 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:44:04.783228 (+ 2436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:44:04.787383 (+ 4155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:44:04.787394 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:44:04.787416 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:44:04.788120 (+ 704us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:44:04.788569 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:44:04.788574 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:44:04.788576 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:44:04.788806 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:44:04.789437 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:44:04.789444 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:44:04.789818 (+ 374us) client_negotiation.cc:769] Sending connection context
0810 18:44:04.790081 (+ 263us) client_negotiation.cc:240] Negotiation successful
0810 18:44:04.790611 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":749,"thread_start_us":469,"threads_started":1}
I20260810 18:44:04.793591 23159 heartbeater.cc:347] Connected to a master server at 127.4.228.254:40945
I20260810 18:44:04.793917 23159 heartbeater.cc:464] Registering TS with master...
I20260810 18:44:04.794823 23159 heartbeater.cc:511] Master 127.4.228.254:40945 requested a full tablet report, sending...
I20260810 18:44:04.796706 22355 ts_manager.cc:194] Registered new tserver with Master: e4231229ed9d4a7fb4d455693f16b399 (127.4.228.195:43777)
I20260810 18:44:04.797575 22355 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.195@KRBTEST.COM'} at 127.4.228.195:45907
I20260810 18:44:04.800851 5011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:44:04.821933 23161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:44:04.806326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51412 (local address 127.4.228.254:40945)
0810 18:44:04.806583 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0810 18:44:04.806588 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:44:04.807472 (+ 884us) server_negotiation.cc:407] Connection header received
0810 18:44:04.807675 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:44:04.807679 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:44:04.807797 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:44:04.807889 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:44:04.809971 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:04.810999 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:04.812737 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:04.813300 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:04.813408 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:44:04.816043 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:44:04.816048 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:44:04.816052 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:44:04.816101 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:44:04.819076 (+ 2975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:44:04.819892 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:44:04.819896 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:44:04.819898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:44:04.820039 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:44:04.820480 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:44:04.820484 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:44:04.820486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:44:04.820813 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:44:04.821006 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0810 18:44:04.821489 (+ 483us) server_negotiation.cc:299] Negotiation successful
0810 18:44:04.821701 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 18:44:04.827253 22355 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51412:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:44:04.835321 22355 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:44:04.879185 23175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:44:04.851529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41478 (local address 127.4.228.195:43777)
0810 18:44:04.866889 (+ 15360us) server_negotiation.cc:206] Beginning negotiation
0810 18:44:04.866903 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:44:04.866948 (+ 45us) server_negotiation.cc:407] Connection header received
0810 18:44:04.867074 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:44:04.867085 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:44:04.867412 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:44:04.867668 (+ 256us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:44:04.868440 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:04.871428 (+ 2988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:04.875597 (+ 4169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:04.877318 (+ 1721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:04.877489 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:44:04.877846 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0810 18:44:04.878624 (+ 778us) server_negotiation.cc:299] Negotiation successful
0810 18:44:04.878813 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15121,"thread_start_us":16481,"threads_started":1}
I20260810 18:44:04.879602 23179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:44:04.851477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:43777 (local address 127.0.0.1:41478)
0810 18:44:04.864379 (+ 12902us) negotiation.cc:107] Waiting for socket to connect
0810 18:44:04.864402 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 18:44:04.864537 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:44:04.867704 (+ 3167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:44:04.867712 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:44:04.867747 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:44:04.868212 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:44:04.868225 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:44:04.871665 (+ 3440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:44:04.871672 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:44:04.875387 (+ 3715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:44:04.875404 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:44:04.878108 (+ 2704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:44:04.878114 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:44:04.878266 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:44:04.878316 (+ 50us) client_negotiation.cc:769] Sending connection context
0810 18:44:04.878553 (+ 237us) client_negotiation.cc:240] Negotiation successful
0810 18:44:04.878789 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12752,"mutex_wait_us":205,"thread_start_us":12960,"threads_started":1}
I20260810 18:44:04.883955 23174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:44:04.851464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:41075 (local address 127.0.0.1:39046)
0810 18:44:04.857476 (+ 6012us) negotiation.cc:107] Waiting for socket to connect
0810 18:44:04.857530 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 18:44:04.857740 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:44:04.873717 (+ 15977us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:44:04.873725 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:44:04.873771 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:44:04.874460 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:44:04.874934 (+ 474us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:44:04.877057 (+ 2123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:44:04.877065 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:44:04.880337 (+ 3272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:44:04.880354 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:44:04.883175 (+ 2821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:44:04.883183 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:44:04.883339 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:44:04.883366 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 18:44:04.883427 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 18:44:04.883525 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5758,"mutex_wait_us":169,"thread_start_us":5575,"threads_started":1}
I20260810 18:44:04.884783 23181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:44:04.854742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:41459 (local address 127.0.0.1:42016)
0810 18:44:04.867101 (+ 12359us) negotiation.cc:107] Waiting for socket to connect
0810 18:44:04.867123 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 18:44:04.867257 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:44:04.871910 (+ 4653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:44:04.871918 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:44:04.871962 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:44:04.873235 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:44:04.873249 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:44:04.877424 (+ 4175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:44:04.877443 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:44:04.881440 (+ 3997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:44:04.881457 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:44:04.883899 (+ 2442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:44:04.883907 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:44:04.884088 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:44:04.884123 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 18:44:04.884219 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 18:44:04.884349 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12219,"mutex_wait_us":4,"thread_start_us":11947,"threads_started":1}
I20260810 18:44:04.885124 23177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:44:04.851795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39046 (local address 127.4.228.193:41075)
0810 18:44:04.873053 (+ 21258us) server_negotiation.cc:206] Beginning negotiation
0810 18:44:04.873063 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:44:04.873096 (+ 33us) server_negotiation.cc:407] Connection header received
0810 18:44:04.873238 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:44:04.873246 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:44:04.873487 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:44:04.874602 (+ 1115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:44:04.875197 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:04.876864 (+ 1667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:04.880586 (+ 3722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:04.882904 (+ 2318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:04.884206 (+ 1302us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:44:04.884447 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0810 18:44:04.884523 (+ 76us) server_negotiation.cc:299] Negotiation successful
0810 18:44:04.884702 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21006,"thread_start_us":20817,"threads_started":1}
I20260810 18:44:04.884687 23061 tablet_service.cc:1467] Processing CreateTablet for tablet c04f6c311bf94021a354ecc0636035ae (DEFAULT_TABLE table=test-table [id=24ebcae2c0954fb394c2ec4eba8216bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:44:04.885704 23182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:44:04.855226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42016 (local address 127.4.228.194:41459)
0810 18:44:04.867647 (+ 12421us) server_negotiation.cc:206] Beginning negotiation
0810 18:44:04.867658 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:44:04.867687 (+ 29us) server_negotiation.cc:407] Connection header received
0810 18:44:04.867778 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:44:04.867788 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:44:04.871618 (+ 3830us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:44:04.871878 (+ 260us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:44:04.873656 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:04.877231 (+ 3575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:04.881669 (+ 4438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:04.883594 (+ 1925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:04.883748 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:44:04.883977 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0810 18:44:04.885097 (+ 1120us) server_negotiation.cc:299] Negotiation successful
0810 18:44:04.885349 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12177,"thread_start_us":11934,"threads_started":1}
I20260810 18:44:04.886927 23061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c04f6c311bf94021a354ecc0636035ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:44:04.889642 22595 tablet_service.cc:1467] Processing CreateTablet for tablet c04f6c311bf94021a354ecc0636035ae (DEFAULT_TABLE table=test-table [id=24ebcae2c0954fb394c2ec4eba8216bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:44:04.889707 22825 tablet_service.cc:1467] Processing CreateTablet for tablet c04f6c311bf94021a354ecc0636035ae (DEFAULT_TABLE table=test-table [id=24ebcae2c0954fb394c2ec4eba8216bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:44:04.890996 22595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c04f6c311bf94021a354ecc0636035ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:44:04.891235 22825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c04f6c311bf94021a354ecc0636035ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:44:04.898620 22930 heartbeater.cc:503] Master 127.4.228.254:40945 was elected leader, sending a full tablet report...
I20260810 18:44:04.906018 23187 tablet_bootstrap.cc:492] T c04f6c311bf94021a354ecc0636035ae P f6565bf8d91346f090f3c3906ebda532: Bootstrap starting.
I20260810 18:44:04.909442 23185 tablet_bootstrap.cc:492] T c04f6c311bf94021a354ecc0636035ae P e4231229ed9d4a7fb4d455693f16b399: Bootstrap starting.
I20260810 18:44:04.909499 23187 tablet_bootstrap.cc:654] T c04f6c311bf94021a354ecc0636035ae P f6565bf8d91346f090f3c3906ebda532: Neither blocks nor log segments found. Creating new log.
I20260810 18:44:04.910929 23187 log.cc:824] T c04f6c311bf94021a354ecc0636035ae P f6565bf8d91346f090f3c3906ebda532: Log is configured to *not* fsync() on all Append() calls
I20260810 18:44:04.911167 23188 tablet_bootstrap.cc:492] T c04f6c311bf94021a354ecc0636035ae P 1c2367a7718142f1bbcd46143de6e9a0: Bootstrap starting.
I20260810 18:44:04.915218 23185 tablet_bootstrap.cc:654] T c04f6c311bf94021a354ecc0636035ae P e4231229ed9d4a7fb4d455693f16b399: Neither blocks nor log segments found. Creating new log.
I20260810 18:44:04.915717 23187 tablet_bootstrap.cc:492] T c04f6c311bf94021a354ecc0636035ae P f6565bf8d91346f090f3c3906ebda532: No bootstrap required, opened a new log
I20260810 18:44:04.916077 23187 ts_tablet_manager.cc:1397] T c04f6c311bf94021a354ecc0636035ae P f6565bf8d91346f090f3c3906ebda532: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260810 18:44:04.916513 23188 tablet_bootstrap.cc:654] T c04f6c311bf94021a354ecc0636035ae P 1c2367a7718142f1bbcd46143de6e9a0: Neither blocks nor log segments found. Creating new log.
I20260810 18:44:04.917172 23185 log.cc:824] T c04f6c311bf94021a354ecc0636035ae P e4231229ed9d4a7fb4d455693f16b399: Log is configured to *not* fsync() on all Append() calls
I20260810 18:44:04.918408 23188 log.cc:824] T c04f6c311bf94021a354ecc0636035ae P 1c2367a7718142f1bbcd46143de6e9a0: Log is configured to *not* fsync() on all Append() calls
I20260810 18:44:04.921900 23185 tablet_bootstrap.cc:492] T c04f6c311bf94021a354ecc0636035ae P e4231229ed9d4a7fb4d455693f16b399: No bootstrap required, opened a new log
I20260810 18:44:04.922338 23185 ts_tablet_manager.cc:1397] T c04f6c311bf94021a354ecc0636035ae P e4231229ed9d4a7fb4d455693f16b399: Time spent bootstrapping tablet: real 0.013s user 0.012s sys 0.000s
I20260810 18:44:04.925901 23188 tablet_bootstrap.cc:492] T c04f6c311bf94021a354ecc0636035ae P 1c2367a7718142f1bbcd46143de6e9a0: No bootstrap required, opened a new log
I20260810 18:44:04.925566 23187 raft_consensus.cc:348] T c04f6c311bf94021a354ecc0636035ae P f6565bf8d91346f090f3c3906ebda532 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4231229ed9d4a7fb4d455693f16b399" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 43777 } } peers { permanent_uuid: "f6565bf8d91346f090f3c3906ebda532" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41075 } } peers { permanent_uuid: "1c2367a7718142f1bbcd46143de6e9a0" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 41459 } }
I20260810 18:44:04.926136 23187 raft_consensus.cc:374] T c04f6c311bf94021a354ecc0636035ae P f6565bf8d91346f090f3c3906ebda532 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:44:04.926270 23187 raft_consensus.cc:729] T c04f6c311bf94021a354ecc0636035ae P f6565bf8d91346f090f3c3906ebda532 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6565bf8d91346f090f3c3906ebda532, State: Initialized, Role: FOLLOWER
I20260810 18:44:04.926353 23188 ts_tablet_manager.cc:1397] T c04f6c311bf94021a354ecc0636035ae P 1c2367a7718142f1bbcd46143de6e9a0: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.003s
I20260810 18:44:04.927114 23187 consensus_queue.cc:260] T c04f6c311bf94021a354ecc0636035ae P f6565bf8d91346f090f3c3906ebda532 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4231229ed9d4a7fb4d455693f16b399" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 43777 } } peers { permanent_uuid: "f6565bf8d91346f090f3c3906ebda532" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41075 } } peers { permanent_uuid: "1c2367a7718142f1bbcd46143de6e9a0" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 41459 } }
I20260810 18:44:04.930265 23187 ts_tablet_manager.cc:1428] T c04f6c311bf94021a354ecc0636035ae P f6565bf8d91346f090f3c3906ebda532: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260810 18:44:04.935997 23185 raft_consensus.cc:348] T c04f6c311bf94021a354ecc0636035ae P e4231229ed9d4a7fb4d455693f16b399 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4231229ed9d4a7fb4d455693f16b399" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 43777 } } peers { permanent_uuid: "f6565bf8d91346f090f3c3906ebda532" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41075 } } peers { permanent_uuid: "1c2367a7718142f1bbcd46143de6e9a0" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 41459 } }
I20260810 18:44:04.936602 23185 raft_consensus.cc:374] T c04f6c311bf94021a354ecc0636035ae P e4231229ed9d4a7fb4d455693f16b399 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:44:04.936743 23185 raft_consensus.cc:729] T c04f6c311bf94021a354ecc0636035ae P e4231229ed9d4a7fb4d455693f16b399 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4231229ed9d4a7fb4d455693f16b399, State: Initialized, Role: FOLLOWER
I20260810 18:44:04.937472 23185 consensus_queue.cc:260] T c04f6c311bf94021a354ecc0636035ae P e4231229ed9d4a7fb4d455693f16b399 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4231229ed9d4a7fb4d455693f16b399" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 43777 } } peers { permanent_uuid: "f6565bf8d91346f090f3c3906ebda532" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41075 } } peers { permanent_uuid: "1c2367a7718142f1bbcd46143de6e9a0" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 41459 } }
I20260810 18:44:04.938835 23159 heartbeater.cc:503] Master 127.4.228.254:40945 was elected leader, sending a full tablet report...
I20260810 18:44:04.939735 23185 ts_tablet_manager.cc:1428] T c04f6c311bf94021a354ecc0636035ae P e4231229ed9d4a7fb4d455693f16b399: Time spent starting tablet: real 0.017s user 0.016s sys 0.000s
I20260810 18:44:04.940150 23188 raft_consensus.cc:348] T c04f6c311bf94021a354ecc0636035ae P 1c2367a7718142f1bbcd46143de6e9a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4231229ed9d4a7fb4d455693f16b399" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 43777 } } peers { permanent_uuid: "f6565bf8d91346f090f3c3906ebda532" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41075 } } peers { permanent_uuid: "1c2367a7718142f1bbcd46143de6e9a0" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 41459 } }
I20260810 18:44:04.940726 23188 raft_consensus.cc:374] T c04f6c311bf94021a354ecc0636035ae P 1c2367a7718142f1bbcd46143de6e9a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:44:04.940871 23188 raft_consensus.cc:729] T c04f6c311bf94021a354ecc0636035ae P 1c2367a7718142f1bbcd46143de6e9a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c2367a7718142f1bbcd46143de6e9a0, State: Initialized, Role: FOLLOWER
I20260810 18:44:04.941511 23188 consensus_queue.cc:260] T c04f6c311bf94021a354ecc0636035ae P 1c2367a7718142f1bbcd46143de6e9a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4231229ed9d4a7fb4d455693f16b399" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 43777 } } peers { permanent_uuid: "f6565bf8d91346f090f3c3906ebda532" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41075 } } peers { permanent_uuid: "1c2367a7718142f1bbcd46143de6e9a0" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 41459 } }
I20260810 18:44:04.943868 23188 ts_tablet_manager.cc:1428] T c04f6c311bf94021a354ecc0636035ae P 1c2367a7718142f1bbcd46143de6e9a0: Time spent starting tablet: real 0.017s user 0.014s sys 0.005s
W20260810 18:44:05.018901 23160 tablet.cc:2369] T c04f6c311bf94021a354ecc0636035ae P e4231229ed9d4a7fb4d455693f16b399: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:44:05.082623 23195 raft_consensus.cc:482] T c04f6c311bf94021a354ecc0636035ae P 1c2367a7718142f1bbcd46143de6e9a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:44:05.083001 23195 raft_consensus.cc:504] T c04f6c311bf94021a354ecc0636035ae P 1c2367a7718142f1bbcd46143de6e9a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4231229ed9d4a7fb4d455693f16b399" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 43777 } } peers { permanent_uuid: "f6565bf8d91346f090f3c3906ebda532" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41075 } } peers { permanent_uuid: "1c2367a7718142f1bbcd46143de6e9a0" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 41459 } }
I20260810 18:44:05.084923 23195 leader_election.cc:288] T c04f6c311bf94021a354ecc0636035ae P 1c2367a7718142f1bbcd46143de6e9a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4231229ed9d4a7fb4d455693f16b399 (127.4.228.195:43777), f6565bf8d91346f090f3c3906ebda532 (127.4.228.193:41075)
W20260810 18:44:05.086458 22698 tablet.cc:2369] T c04f6c311bf94021a354ecc0636035ae P f6565bf8d91346f090f3c3906ebda532: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:44:05.093896 23175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:44:05.085573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:56561 (local address 127.4.228.195:43777)
0810 18:44:05.085852 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0810 18:44:05.085859 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:44:05.086780 (+ 921us) server_negotiation.cc:407] Connection header received
0810 18:44:05.087236 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:44:05.087243 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:44:05.087347 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:44:05.087464 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:44:05.088457 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:05.089270 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:05.091261 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:05.092637 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:05.092799 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:44:05.092861 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 18:44:05.093354 (+ 493us) server_negotiation.cc:299] Negotiation successful
0810 18:44:05.093545 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260810 18:44:05.093918 23202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:44:05.085284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:43777 (local address 127.4.228.194:56561)
0810 18:44:05.086609 (+ 1325us) negotiation.cc:107] Waiting for socket to connect
0810 18:44:05.086646 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 18:44:05.087001 (+ 355us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:44:05.087560 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:44:05.087564 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:44:05.087594 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:44:05.088278 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:44:05.088294 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:44:05.089408 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:44:05.089414 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:44:05.091061 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:44:05.091077 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:44:05.092867 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:44:05.092873 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:44:05.093057 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:44:05.093129 (+ 72us) client_negotiation.cc:769] Sending connection context
0810 18:44:05.093294 (+ 165us) client_negotiation.cc:240] Negotiation successful
0810 18:44:05.093435 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1086,"spinlock_wait_cycles":18816,"thread_start_us":1114,"threads_started":1}
I20260810 18:44:05.094384 23177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:44:05.085836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:36251 (local address 127.4.228.193:41075)
0810 18:44:05.086096 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0810 18:44:05.086103 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:44:05.087379 (+ 1276us) server_negotiation.cc:407] Connection header received
0810 18:44:05.087604 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:44:05.087608 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:44:05.087678 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:44:05.087791 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:44:05.088775 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:05.089557 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:05.092089 (+ 2532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:05.093063 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:05.093306 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:44:05.093463 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 18:44:05.093968 (+ 505us) server_negotiation.cc:299] Negotiation successful
0810 18:44:05.094126 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260810 18:44:05.095082 23199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:44:05.085397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:41075 (local address 127.4.228.194:36251)
0810 18:44:05.087238 (+ 1841us) negotiation.cc:107] Waiting for socket to connect
0810 18:44:05.087254 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:44:05.087433 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:44:05.087814 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:44:05.087819 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:44:05.087882 (+ 63us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:44:05.088577 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:44:05.088587 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:44:05.089727 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:44:05.089734 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:44:05.091869 (+ 2135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:44:05.091891 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:44:05.093385 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:44:05.093393 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:44:05.093755 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:44:05.093791 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 18:44:05.093938 (+ 147us) client_negotiation.cc:240] Negotiation successful
0810 18:44:05.094512 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1691,"thread_start_us":1210,"threads_started":1}
I20260810 18:44:05.095429 23086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c04f6c311bf94021a354ecc0636035ae" candidate_uuid: "1c2367a7718142f1bbcd46143de6e9a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4231229ed9d4a7fb4d455693f16b399" is_pre_election: true
I20260810 18:44:05.096298 23086 raft_consensus.cc:2454] T c04f6c311bf94021a354ecc0636035ae P e4231229ed9d4a7fb4d455693f16b399 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c2367a7718142f1bbcd46143de6e9a0 in term 0.
I20260810 18:44:05.096563 22628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c04f6c311bf94021a354ecc0636035ae" candidate_uuid: "1c2367a7718142f1bbcd46143de6e9a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6565bf8d91346f090f3c3906ebda532" is_pre_election: true
I20260810 18:44:05.097471 22752 leader_election.cc:302] T c04f6c311bf94021a354ecc0636035ae P 1c2367a7718142f1bbcd46143de6e9a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c2367a7718142f1bbcd46143de6e9a0, e4231229ed9d4a7fb4d455693f16b399; no voters:
I20260810 18:44:05.097715 22628 raft_consensus.cc:2454] T c04f6c311bf94021a354ecc0636035ae P f6565bf8d91346f090f3c3906ebda532 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c2367a7718142f1bbcd46143de6e9a0 in term 0.
I20260810 18:44:05.098075 23195 raft_consensus.cc:2781] T c04f6c311bf94021a354ecc0636035ae P 1c2367a7718142f1bbcd46143de6e9a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:44:05.098347 23195 raft_consensus.cc:482] T c04f6c311bf94021a354ecc0636035ae P 1c2367a7718142f1bbcd46143de6e9a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:44:05.098611 23195 raft_consensus.cc:3037] T c04f6c311bf94021a354ecc0636035ae P 1c2367a7718142f1bbcd46143de6e9a0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:44:05.101959 23195 raft_consensus.cc:504] T c04f6c311bf94021a354ecc0636035ae P 1c2367a7718142f1bbcd46143de6e9a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4231229ed9d4a7fb4d455693f16b399" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 43777 } } peers { permanent_uuid: "f6565bf8d91346f090f3c3906ebda532" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41075 } } peers { permanent_uuid: "1c2367a7718142f1bbcd46143de6e9a0" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 41459 } }
I20260810 18:44:05.102941 23195 leader_election.cc:288] T c04f6c311bf94021a354ecc0636035ae P 1c2367a7718142f1bbcd46143de6e9a0 [CANDIDATE]: Term 1 election: Requested vote from peers e4231229ed9d4a7fb4d455693f16b399 (127.4.228.195:43777), f6565bf8d91346f090f3c3906ebda532 (127.4.228.193:41075)
I20260810 18:44:05.103554 22628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c04f6c311bf94021a354ecc0636035ae" candidate_uuid: "1c2367a7718142f1bbcd46143de6e9a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6565bf8d91346f090f3c3906ebda532"
I20260810 18:44:05.103588 23086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c04f6c311bf94021a354ecc0636035ae" candidate_uuid: "1c2367a7718142f1bbcd46143de6e9a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4231229ed9d4a7fb4d455693f16b399"
I20260810 18:44:05.103828 23086 raft_consensus.cc:3037] T c04f6c311bf94021a354ecc0636035ae P e4231229ed9d4a7fb4d455693f16b399 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:44:05.103820 22628 raft_consensus.cc:3037] T c04f6c311bf94021a354ecc0636035ae P f6565bf8d91346f090f3c3906ebda532 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:44:05.108352 22628 raft_consensus.cc:2454] T c04f6c311bf94021a354ecc0636035ae P f6565bf8d91346f090f3c3906ebda532 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c2367a7718142f1bbcd46143de6e9a0 in term 1.
I20260810 18:44:05.108352 23086 raft_consensus.cc:2454] T c04f6c311bf94021a354ecc0636035ae P e4231229ed9d4a7fb4d455693f16b399 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c2367a7718142f1bbcd46143de6e9a0 in term 1.
I20260810 18:44:05.108999 22751 leader_election.cc:302] T c04f6c311bf94021a354ecc0636035ae P 1c2367a7718142f1bbcd46143de6e9a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1c2367a7718142f1bbcd46143de6e9a0, e4231229ed9d4a7fb4d455693f16b399, f6565bf8d91346f090f3c3906ebda532; no voters:
I20260810 18:44:05.109503 23195 raft_consensus.cc:2781] T c04f6c311bf94021a354ecc0636035ae P 1c2367a7718142f1bbcd46143de6e9a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:44:05.110366 23195 raft_consensus.cc:686] T c04f6c311bf94021a354ecc0636035ae P 1c2367a7718142f1bbcd46143de6e9a0 [term 1 LEADER]: Becoming Leader. State: Replica: 1c2367a7718142f1bbcd46143de6e9a0, State: Running, Role: LEADER
I20260810 18:44:05.111157 23195 consensus_queue.cc:237] T c04f6c311bf94021a354ecc0636035ae P 1c2367a7718142f1bbcd46143de6e9a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4231229ed9d4a7fb4d455693f16b399" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 43777 } } peers { permanent_uuid: "f6565bf8d91346f090f3c3906ebda532" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41075 } } peers { permanent_uuid: "1c2367a7718142f1bbcd46143de6e9a0" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 41459 } }
W20260810 18:44:05.115370 22931 tablet.cc:2369] T c04f6c311bf94021a354ecc0636035ae P 1c2367a7718142f1bbcd46143de6e9a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:44:05.117338 22353 catalog_manager.cc:5495] T c04f6c311bf94021a354ecc0636035ae P 1c2367a7718142f1bbcd46143de6e9a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c2367a7718142f1bbcd46143de6e9a0 (127.4.228.194). New cstate: current_term: 1 leader_uuid: "1c2367a7718142f1bbcd46143de6e9a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4231229ed9d4a7fb4d455693f16b399" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 43777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6565bf8d91346f090f3c3906ebda532" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c2367a7718142f1bbcd46143de6e9a0" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 41459 } health_report { overall_health: HEALTHY } } }
I20260810 18:44:05.177946 23182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:44:05.170687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42026 (local address 127.4.228.194:41459)
0810 18:44:05.170962 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0810 18:44:05.170969 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:44:05.171085 (+ 116us) server_negotiation.cc:407] Connection header received
0810 18:44:05.171360 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:44:05.171364 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:44:05.171459 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:44:05.171626 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:44:05.172173 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:05.173153 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:05.175197 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:05.175787 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:05.175933 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:44:05.176621 (+ 688us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:44:05.177230 (+ 609us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:44:05.177400 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 18:44:05.177483 (+ 83us) server_negotiation.cc:299] Negotiation successful
0810 18:44:05.177576 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260810 18:44:05.189743 22628 raft_consensus.cc:1261] T c04f6c311bf94021a354ecc0636035ae P f6565bf8d91346f090f3c3906ebda532 [term 1 FOLLOWER]: Refusing update from remote peer 1c2367a7718142f1bbcd46143de6e9a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:44:05.189735 23086 raft_consensus.cc:1261] T c04f6c311bf94021a354ecc0636035ae P e4231229ed9d4a7fb4d455693f16b399 [term 1 FOLLOWER]: Refusing update from remote peer 1c2367a7718142f1bbcd46143de6e9a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:44:05.191313 23203 consensus_queue.cc:1035] T c04f6c311bf94021a354ecc0636035ae P 1c2367a7718142f1bbcd46143de6e9a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6565bf8d91346f090f3c3906ebda532" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:44:05.191967 23195 consensus_queue.cc:1035] T c04f6c311bf94021a354ecc0636035ae P 1c2367a7718142f1bbcd46143de6e9a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4231229ed9d4a7fb4d455693f16b399" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 43777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:44:05.230758 22353 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:44:05.231190 22353 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:44:05.236688 22353 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4b6da28a44ba42aeb88c59fb8b7401a4: Sending DeleteTablet for 3 replicas of tablet c04f6c311bf94021a354ecc0636035ae
I20260810 18:44:05.238363 23061 tablet_service.cc:1514] Processing DeleteTablet for tablet c04f6c311bf94021a354ecc0636035ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:44:05 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:41478
I20260810 18:44:05.238451 22595 tablet_service.cc:1514] Processing DeleteTablet for tablet c04f6c311bf94021a354ecc0636035ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:44:05 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:39046
I20260810 18:44:05.239562 22825 tablet_service.cc:1514] Processing DeleteTablet for tablet c04f6c311bf94021a354ecc0636035ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:44:05 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:42016
I20260810 18:44:05.244247 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 22490
I20260810 18:44:05.253991 23226 tablet_replica.cc:330] T c04f6c311bf94021a354ecc0636035ae P e4231229ed9d4a7fb4d455693f16b399: stopping tablet replica
I20260810 18:44:05.254642 23226 raft_consensus.cc:2229] T c04f6c311bf94021a354ecc0636035ae P e4231229ed9d4a7fb4d455693f16b399 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:44:05.255312 23226 raft_consensus.cc:2258] T c04f6c311bf94021a354ecc0636035ae P e4231229ed9d4a7fb4d455693f16b399 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:44:05.256326 23227 tablet_replica.cc:330] T c04f6c311bf94021a354ecc0636035ae P f6565bf8d91346f090f3c3906ebda532: stopping tablet replica
I20260810 18:44:05.256909 23227 raft_consensus.cc:2229] T c04f6c311bf94021a354ecc0636035ae P f6565bf8d91346f090f3c3906ebda532 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:44:05.257565 23227 raft_consensus.cc:2258] T c04f6c311bf94021a354ecc0636035ae P f6565bf8d91346f090f3c3906ebda532 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:44:05.260851 23227 ts_tablet_manager.cc:1905] T c04f6c311bf94021a354ecc0636035ae P f6565bf8d91346f090f3c3906ebda532: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:44:05.265235 23228 tablet_replica.cc:330] T c04f6c311bf94021a354ecc0636035ae P 1c2367a7718142f1bbcd46143de6e9a0: stopping tablet replica
I20260810 18:44:05.265834 23228 raft_consensus.cc:2229] T c04f6c311bf94021a354ecc0636035ae P 1c2367a7718142f1bbcd46143de6e9a0 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:44:05.266847 23228 raft_consensus.cc:2258] T c04f6c311bf94021a354ecc0636035ae P 1c2367a7718142f1bbcd46143de6e9a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:44:05.269294 23226 ts_tablet_manager.cc:1905] T c04f6c311bf94021a354ecc0636035ae P e4231229ed9d4a7fb4d455693f16b399: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:44:05.269304 23228 ts_tablet_manager.cc:1905] T c04f6c311bf94021a354ecc0636035ae P 1c2367a7718142f1bbcd46143de6e9a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:44:05.276003 23227 ts_tablet_manager.cc:1918] T c04f6c311bf94021a354ecc0636035ae P f6565bf8d91346f090f3c3906ebda532: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:44:05.276324 23227 log.cc:1196] T c04f6c311bf94021a354ecc0636035ae P f6565bf8d91346f090f3c3906ebda532: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-0/wal/wals/c04f6c311bf94021a354ecc0636035ae
I20260810 18:44:05.277160 23227 ts_tablet_manager.cc:1939] T c04f6c311bf94021a354ecc0636035ae P f6565bf8d91346f090f3c3906ebda532: Deleting consensus metadata
I20260810 18:44:05.277501 23228 ts_tablet_manager.cc:1918] T c04f6c311bf94021a354ecc0636035ae P 1c2367a7718142f1bbcd46143de6e9a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:44:05.277503 23226 ts_tablet_manager.cc:1918] T c04f6c311bf94021a354ecc0636035ae P e4231229ed9d4a7fb4d455693f16b399: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:44:05.277683 23228 log.cc:1196] T c04f6c311bf94021a354ecc0636035ae P 1c2367a7718142f1bbcd46143de6e9a0: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-1/wal/wals/c04f6c311bf94021a354ecc0636035ae
I20260810 18:44:05.277676 23226 log.cc:1196] T c04f6c311bf94021a354ecc0636035ae P e4231229ed9d4a7fb4d455693f16b399: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387267874959-5011-0/minicluster-data/ts-2/wal/wals/c04f6c311bf94021a354ecc0636035ae
I20260810 18:44:05.278232 23226 ts_tablet_manager.cc:1939] T c04f6c311bf94021a354ecc0636035ae P e4231229ed9d4a7fb4d455693f16b399: Deleting consensus metadata
I20260810 18:44:05.278249 23228 ts_tablet_manager.cc:1939] T c04f6c311bf94021a354ecc0636035ae P 1c2367a7718142f1bbcd46143de6e9a0: Deleting consensus metadata
I20260810 18:44:05.280556 22336 catalog_manager.cc:4841] TS e4231229ed9d4a7fb4d455693f16b399 (127.4.228.195:43777): tablet c04f6c311bf94021a354ecc0636035ae (table test-table [id=24ebcae2c0954fb394c2ec4eba8216bd]) successfully deleted
I20260810 18:44:05.280910 22333 catalog_manager.cc:4841] TS 1c2367a7718142f1bbcd46143de6e9a0 (127.4.228.194:41459): tablet c04f6c311bf94021a354ecc0636035ae (table test-table [id=24ebcae2c0954fb394c2ec4eba8216bd]) successfully deleted
Aug 10 18:44:05 dist-test-slave-7kzw krb5kdc[21601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.193@KRBTEST.COM
I20260810 18:44:05.283617 22335 catalog_manager.cc:4841] TS f6565bf8d91346f090f3c3906ebda532 (127.4.228.193:41075): tablet c04f6c311bf94021a354ecc0636035ae (table test-table [id=24ebcae2c0954fb394c2ec4eba8216bd]) successfully deleted
I20260810 18:44:05.290753 23177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:44:05.247045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39056 (local address 127.4.228.193:41075)
0810 18:44:05.247328 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0810 18:44:05.247339 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:44:05.261917 (+ 14578us) server_negotiation.cc:407] Connection header received
0810 18:44:05.262445 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:44:05.262452 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:44:05.262560 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:44:05.262649 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:44:05.265631 (+ 2982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:05.267307 (+ 1676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:05.270715 (+ 3408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:05.271602 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:05.271734 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:44:05.283412 (+ 11678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:44:05.283430 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:44:05.283446 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:44:05.283497 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:44:05.287023 (+ 3526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:44:05.288032 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:44:05.288042 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:44:05.288044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:44:05.288214 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:44:05.288662 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:44:05.288668 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:44:05.288672 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:44:05.289106 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:44:05.289607 (+ 501us) server_negotiation.cc:1035] Waiting for connection context
0810 18:44:05.289948 (+ 341us) server_negotiation.cc:299] Negotiation successful
0810 18:44:05.290244 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 18:44:05.292013 22688 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:44:05.461004 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 22490
I20260810 18:44:05.495137 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 22711
Aug 10 18:44:05 dist-test-slave-7kzw krb5kdc[21601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.194@KRBTEST.COM
I20260810 18:44:05.514601 23182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:44:05.496033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42032 (local address 127.4.228.194:41459)
0810 18:44:05.496248 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 18:44:05.496253 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:44:05.496277 (+ 24us) server_negotiation.cc:407] Connection header received
0810 18:44:05.496499 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:44:05.496503 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:44:05.496630 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:44:05.496721 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:44:05.498626 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:05.499471 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:05.501487 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:05.502256 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:05.502389 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:44:05.507851 (+ 5462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:44:05.507868 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:44:05.507888 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:44:05.507939 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:44:05.511366 (+ 3427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:44:05.512173 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:44:05.512181 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:44:05.512183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:44:05.512356 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:44:05.512784 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:44:05.512789 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:44:05.512792 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:44:05.513197 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:44:05.513549 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0810 18:44:05.513878 (+ 329us) server_negotiation.cc:299] Negotiation successful
0810 18:44:05.514146 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260810 18:44:05.515417 22921 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:44:05.633538 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 22711
I20260810 18:44:05.670639 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 22940
Aug 10 18:44:05 dist-test-slave-7kzw krb5kdc[21601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.195@KRBTEST.COM
I20260810 18:44:05.688283 23236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:44:05.671691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41482 (local address 127.4.228.195:43777)
0810 18:44:05.672259 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0810 18:44:05.672263 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:44:05.672283 (+ 20us) server_negotiation.cc:407] Connection header received
0810 18:44:05.672345 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:44:05.672348 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:44:05.672420 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:44:05.672525 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:44:05.674077 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:05.675024 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:05.676524 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:05.676975 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:05.677087 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:44:05.682152 (+ 5065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:44:05.682169 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:44:05.682189 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:44:05.682230 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:44:05.685333 (+ 3103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:44:05.686027 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:44:05.686036 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:44:05.686038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:44:05.686212 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:44:05.686634 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:44:05.686639 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:44:05.686643 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:44:05.687036 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:44:05.687311 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0810 18:44:05.687743 (+ 432us) server_negotiation.cc:299] Negotiation successful
0810 18:44:05.687938 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":313,"threads_started":1}
I20260810 18:44:05.689097 23148 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:44:05.784269 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 22940
I20260810 18:44:05.812017 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 22298
I20260810 18:44:05.813493 22407 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:44:05.931180 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 22298
I20260810 18:44:05.966017 5011 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 18:44:06.313427 5011 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 18:44:06.313779 5011 mini_ranger.cc:66] Stopping Ranger...
I20260810 18:44:06.661291 5011 mini_ranger.cc:68] Stopped Ranger
2026-08-10 18:44:06.661 UTC [21646] LOG: received smart shutdown request
2026-08-10 18:44:06.665 UTC [21646] LOG: background worker "logical replication launcher" (PID 21659) exited with exit code 1
2026-08-10 18:44:06.666 UTC [21651] LOG: shutting down
2026-08-10 18:44:06.858 UTC [21646] LOG: database system is shut down
2026-08-10T18:44:06Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (58561 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:44:07 dist-test-slave-7kzw krb5kdc[23257](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:44:07 dist-test-slave-7kzw krb5kdc[23257](info): set up 2 sockets
Aug 10 18:44:07 dist-test-slave-7kzw krb5kdc[23257](info): commencing operation
krb5kdc: starting...
W20260810 18:44:08.382674 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.334s user 0.005s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:44:08 dist-test-slave-7kzw krb5kdc[23257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:44:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:44:08Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.4.228.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.4.228.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.4.228.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/rangeradmin_127.4.228.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.4.228.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/rangeradmin_127.4.228.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.4.228.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.4.228.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.4.228.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/rangerlookup_127.4.228.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.4.228.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/rangerlookup_127.4.228.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.228.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.4.228.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.228.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/HTTP_127.4.228.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.228.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/HTTP_127.4.228.212@KRBTEST.COM.keytab.
I20260810 18:44:08.814170 5011 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskTAuB_V/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/postgres -l logfile start
2026-08-10 18:44:11.606 UTC [23305] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 18:44:11.606 UTC [23305] LOG: listening on IPv4 address "127.4.228.212", port 45645
2026-08-10 18:44:11.611 UTC [23305] LOG: listening on Unix socket "/tmp/.s.PGSQL.45645"
2026-08-10 18:44:11.636 UTC [23308] LOG: database system was shut down at 2026-08-10 18:44:09 UTC
2026-08-10 18:44:11.639 UTC [23305] LOG: database system is ready to accept connections
I20260810 18:44:12.912134 5011 mini_postgres.cc:96] Postgres bound to 45645
2026-08-10 18:44:12.937 UTC [23319] FATAL: database "slave" does not exist
127.4.228.212:45645 - accepting connections
I20260810 18:44:12.938020 5011 mini_ranger.cc:161] Starting Ranger...
I20260810 18:44:12.972913 5011 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 18:44:13.541123 5011 mini_ranger.cc:87] Created ranger Postgres database
I20260810 18:44:13.541260 5011 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ranger-admin
2026-08-10 18:44:13,854 [I] DB FLAVOR :POSTGRES
2026-08-10 18:44:13,854 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 18:44:13,855 [I] Checking connection..
2026-08-10 18:44:13,855 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:45645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 18:44:14,134 [I] Checking connection passed.
2026-08-10 18:44:14,134 [I] --------- Verifying version history table ---------
2026-08-10 18:44:14,134 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:45645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 18:44:14,412 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 18:44:14,412 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:45645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 18:44:14,682 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 18:44:14,682 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 18:44:14,682 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:45645/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 18:44:14.936 UTC [23418] WARNING: there is no transaction in progress
2026-08-10 18:44:14,943 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 18:44:14,944 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:45645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 18:44:15,218 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 18:44:15,218 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 18:44:15,219 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:45645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 18:44:15,489 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:45645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 18:44:15,754 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:45645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-7kzw.c.gcp-upstream.internal','N') ;"
2026-08-10 18:44:16,020 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:45645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 18:44:16,296 [I] Table x_portal_user does not exist in database ranger
2026-08-10 18:44:16,297 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:45645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 18:44:16,554 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 18:44:16,554 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:45645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 18:44:16,797 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:45645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 18:44:17,089 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 18:44:17,089 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:45645/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 18:44:18.144 UTC [23597] WARNING: there is no transaction in progress
2026-08-10 18:44:18.161 UTC [23597] WARNING: there is no transaction in progress
2026-08-10 18:44:18.178 UTC [23597] WARNING: there is no transaction in progress
2026-08-10 18:44:18.194 UTC [23597] WARNING: there is no transaction in progress
2026-08-10 18:44:18.211 UTC [23597] WARNING: there is no transaction in progress
2026-08-10 18:44:18.231 UTC [23597] WARNING: there is no transaction in progress
2026-08-10 18:44:18.289 UTC [23597] WARNING: there is no transaction in progress
2026-08-10 18:44:18.296 UTC [23597] WARNING: there is no transaction in progress
2026-08-10 18:44:18.311 UTC [23597] WARNING: there is no transaction in progress
2026-08-10 18:44:18.321 UTC [23597] WARNING: there is no transaction in progress
2026-08-10 18:44:18.332 UTC [23597] WARNING: there is no transaction in progress
2026-08-10 18:44:18.346 UTC [23597] WARNING: there is no transaction in progress
2026-08-10 18:44:18.355 UTC [23597] WARNING: there is no transaction in progress
2026-08-10 18:44:18.362 UTC [23597] WARNING: there is no transaction in progress
2026-08-10 18:44:18.369 UTC [23597] WARNING: there is no transaction in progress
2026-08-10 18:44:18.376 UTC [23597] WARNING: there is no transaction in progress
2026-08-10 18:44:19,104 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 18:44:19,104 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:45645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 18:44:19,350 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 18:44:19,350 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:45645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 18:44:19,642 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 18:44:19,643 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:45645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 18:44:19,929 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 18:44:19,929 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:45645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 18:44:20,176 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:45645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 18:44:20,472 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:45645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-7kzw.c.gcp-upstream.internal';"
2026-08-10 18:44:20,722 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 18:44:20,723 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:45645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 18:44:20,975 [I] DB_PATCHES have already been applied
I20260810 18:44:20.981940 5011 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ranger-admin:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskTAuB_V/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 18:44:20.982061 5011 mini_ranger.cc:193] Using host: 127.4.228.212
I20260810 18:44:21.003356 5011 mini_ranger.cc:227] Ranger admin URL: http://127.4.228.212:44535
Aug 10, 2026 6:44:21 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 6:44:21 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 6:44:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 6:44:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 6:44:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 6:44:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 6:44:21 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 18:44:22.529891 5011 mini_ranger.cc:160] Time spent starting Ranger: real 9.592s user 0.007s sys 0.059s
Aug 10, 2026 6:44:43 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 6:44:43 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 10, 2026 6:44:43 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 6:44:43 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 6:44:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 6:44:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 6:44:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 6:44:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 6:44:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 6:44:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 6:44:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 6:44:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 6:44:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 6:44:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 6:44:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 6:44:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 6:44:43 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 6:44:44 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 18:44:45.093045 5011 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.4.228.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.4.228.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.4.228.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/rangerkms_127.4.228.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.4.228.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/rangerkms_127.4.228.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.228.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.4.228.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.228.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/HTTP_127.4.228.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.228.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/HTTP_127.4.228.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 18:44:45 dist-test-slave-7kzw krb5kdc[23257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387485, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 18:44:45.291215 5011 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 18:44:45.318784 5011 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 18:44:46.238636 5011 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 18:44:46.238786 5011 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ranger-kms
I20260810 18:44:46.238852 5011 mini_ranger_kms.cc:227] Using postgres at 127.4.228.212:45645
2026-08-10 18:44:46,640 [I] DB FLAVOR :POSTGRES
2026-08-10 18:44:46,640 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 18:44:46,640 [I] Checking connection
2026-08-10 18:44:46,641 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:45645/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 18:44:46,882 [I] connection success
2026-08-10 18:44:46,882 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 18:44:46,882 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 18:44:46,882 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:45645/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 18:44:47,126 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 18:44:47,126 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 18:44:47,126 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 18:44:47,126 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.228.212:45645/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 18:44:47,404 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 18:44:47.931692 5011 mini_ranger_kms.cc:310] Created kms service
I20260810 18:44:48.125450 5011 mini_ranger_kms.cc:326] Created kudu user
I20260810 18:44:48.175096 5011 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 18:44:48 dist-test-slave-7kzw krb5kdc[23257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387485, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.4.228.212@KRBTEST.COM
I20260810 18:44:48.606197 5011 mini_ranger_kms.cc:384] Added ranger policy
I20260810 18:44:48.606626 5011 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ranger-kms:/tmp/dist-test-taskTAuB_V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskTAuB_V/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 18:44:48.606689 5011 mini_ranger_kms.cc:240] Using host: 127.4.228.212
I20260810 18:44:48.633213 5011 mini_ranger_kms.cc:276] Ranger KMS PID: 23870
I20260810 18:44:48.633370 5011 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.4.228.212:34735
Aug 10, 2026 6:44:49 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 6:44:49 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 6:44:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 6:44:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 6:44:50 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 6:44:50 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 18:44:51 dist-test-slave-7kzw krb5kdc[23257](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786387491, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.228.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:44:51.489017 5011 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.198s user 0.074s sys 0.363s
I20260810 18:44:51.489326 5011 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 18:44:51.489521 5011 mini_ranger_kms.cc:401] 127.4.228.212:34735/kms/v1/keys
Aug 10 18:44:51 dist-test-slave-7kzw krb5kdc[23257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387491, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.228.212@KRBTEST.COM for HTTP/127.4.228.212@KRBTEST.COM
Aug 10 18:44:51 dist-test-slave-7kzw krb5kdc[23257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387491, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.228.212@KRBTEST.COM for HTTP/127.4.228.212@KRBTEST.COM
Aug 10 18:44:54 dist-test-slave-7kzw krb5kdc[23257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387494, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.4.228.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
I20260810 18:44:54.275465 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:33847
--webserver_interface=127.4.228.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:32917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:33847
--ranger_config_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.228.212:34735/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:44:54.634631 23947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:44:54.634933 23947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:44:54.635083 23947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:44:54.642362 23947 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:44:54.642542 23947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:44:54.642659 23947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:44:54.642767 23947 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:44:54.642848 23947 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:44:54.653364 23947 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:32917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/master-0/wal
--ranger_kms_url=127.4.228.212:34735/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:33847
--ranger_config_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:33847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:44:54.654863 23947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:44:54.656302 23947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:44:54.670665 23956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:44:54.670724 23959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:44:54.670991 23965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:44:54.671576 23947 server_base.cc:1034] running on GCE node
I20260810 18:44:54.672596 23947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:44:54.674232 23947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:44:54.675462 23947 hybrid_clock.cc:648] HybridClock initialized: now 1786387494675393 us; error 64 us; skew 500 ppm
I20260810 18:44:54.675856 23947 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:44:54 dist-test-slave-7kzw krb5kdc[23257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387494, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:44:54.679216 23947 init.cc:377] Logged in from keytab as kudu/127.4.228.254@KRBTEST.COM (short username kudu)
I20260810 18:44:54.680928 23947 webserver.cc:460] Webserver started at http://127.4.228.254:34417/ using document root <none> and password file <none>
I20260810 18:44:54.681629 23947 fs_manager.cc:359] Metadata directory not provided
I20260810 18:44:54.681763 23947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:44:54.682113 23947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 18:44:54 dist-test-slave-7kzw krb5kdc[23257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387494, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for HTTP/127.4.228.212@KRBTEST.COM
I20260810 18:44:54.740938 23947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "e41cf7c93c134bccbda0e0982bf85b37"
format_stamp: "Formatted at 2026-08-10 18:44:54 on dist-test-slave-7kzw"
server_key: "ccc7a12e1bbca2dfa8d1a7c4167f57af"
server_key_iv: "608d5747c5aa86def6a3b7aa4d739b33"
server_key_version: "kuduclusterkey@0"
I20260810 18:44:54.742087 23947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "e41cf7c93c134bccbda0e0982bf85b37"
format_stamp: "Formatted at 2026-08-10 18:44:54 on dist-test-slave-7kzw"
server_key: "ccc7a12e1bbca2dfa8d1a7c4167f57af"
server_key_iv: "608d5747c5aa86def6a3b7aa4d739b33"
server_key_version: "kuduclusterkey@0"
I20260810 18:44:54.749262 23947 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260810 18:44:54.769851 23977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:44:54.771835 23947 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260810 18:44:54.772069 23947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "e41cf7c93c134bccbda0e0982bf85b37"
format_stamp: "Formatted at 2026-08-10 18:44:54 on dist-test-slave-7kzw"
server_key: "ccc7a12e1bbca2dfa8d1a7c4167f57af"
server_key_iv: "608d5747c5aa86def6a3b7aa4d739b33"
server_key_version: "kuduclusterkey@0"
I20260810 18:44:54.772357 23947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:44:54.788776 23947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:44:54.793262 23947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:44:54.793610 23947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:44:54.814261 23947 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:33847
I20260810 18:44:54.814301 24056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:33847 every 8 connection(s)
I20260810 18:44:54.815838 23947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:44:54.819582 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 23947
I20260810 18:44:54.819902 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:44:54.820676 5011 external_mini_cluster.cc:1428] Setting key e6ed8b04319688f582fb8dee3c557d85
I20260810 18:44:54.820698 24058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:44:54.835665 24058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37: Bootstrap starting.
I20260810 18:44:54.839906 24058 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37: Neither blocks nor log segments found. Creating new log.
I20260810 18:44:54.841255 24058 log.cc:824] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37: Log is configured to *not* fsync() on all Append() calls
I20260810 18:44:54.844637 24058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37: No bootstrap required, opened a new log
Aug 10 18:44:54 dist-test-slave-7kzw krb5kdc[23257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387494, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:44:54.854919 24058 raft_consensus.cc:348] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41cf7c93c134bccbda0e0982bf85b37" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 33847 } }
I20260810 18:44:54.855396 24058 raft_consensus.cc:374] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:44:54.855540 24058 raft_consensus.cc:729] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e41cf7c93c134bccbda0e0982bf85b37, State: Initialized, Role: FOLLOWER
I20260810 18:44:54.856359 24058 consensus_queue.cc:260] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41cf7c93c134bccbda0e0982bf85b37" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 33847 } }
I20260810 18:44:54.856652 24058 raft_consensus.cc:388] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:44:54.856909 24058 raft_consensus.cc:482] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:44:54.857147 24058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:44:54.860322 24058 raft_consensus.cc:504] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41cf7c93c134bccbda0e0982bf85b37" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 33847 } }
I20260810 18:44:54.861061 24058 leader_election.cc:302] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e41cf7c93c134bccbda0e0982bf85b37; no voters:
I20260810 18:44:54.863268 24062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:44:54.823229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44120 (local address 127.4.228.254:33847)
0810 18:44:54.837600 (+ 14371us) server_negotiation.cc:206] Beginning negotiation
0810 18:44:54.837615 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 18:44:54.837671 (+ 56us) server_negotiation.cc:407] Connection header received
0810 18:44:54.839726 (+ 2055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:44:54.839753 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:44:54.840268 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:44:54.840733 (+ 465us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:44:54.842575 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:54.844597 (+ 2022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:54.846552 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:54.847843 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:54.848047 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:44:54.853399 (+ 5352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:44:54.853412 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:44:54.853439 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:44:54.853492 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:44:54.857667 (+ 4175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:44:54.858753 (+ 1086us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:44:54.858763 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:44:54.858787 (+ 24us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:44:54.859028 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:44:54.859556 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:44:54.859562 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:44:54.859565 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:44:54.860343 (+ 778us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:44:54.860621 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0810 18:44:54.861349 (+ 728us) server_negotiation.cc:299] Negotiation successful
0810 18:44:54.861792 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14097,"thread_start_us":15133,"threads_started":1}
I20260810 18:44:54.865303 24058 leader_election.cc:288] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:44:54.865412 24067 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:44:54.869143 24067 raft_consensus.cc:686] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37 [term 1 LEADER]: Becoming Leader. State: Replica: e41cf7c93c134bccbda0e0982bf85b37, State: Running, Role: LEADER
I20260810 18:44:54.869794 24067 consensus_queue.cc:237] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41cf7c93c134bccbda0e0982bf85b37" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 33847 } }
I20260810 18:44:54.870026 24058 sys_catalog.cc:564] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:44:54.874996 24068 sys_catalog.cc:455] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e41cf7c93c134bccbda0e0982bf85b37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41cf7c93c134bccbda0e0982bf85b37" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 33847 } } }
I20260810 18:44:54.875537 24068 sys_catalog.cc:458] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37 [sys.catalog]: This master's current role is: LEADER
I20260810 18:44:54.876139 24072 sys_catalog.cc:455] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e41cf7c93c134bccbda0e0982bf85b37. Latest consensus state: current_term: 1 leader_uuid: "e41cf7c93c134bccbda0e0982bf85b37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41cf7c93c134bccbda0e0982bf85b37" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 33847 } } }
I20260810 18:44:54.876520 24072 sys_catalog.cc:458] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37 [sys.catalog]: This master's current role is: LEADER
I20260810 18:44:54.890942 24076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:44:54.897701 24076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:44:54.924810 24076 catalog_manager.cc:1347] Generated new cluster ID: 7c481587ca70453b83351ce0322bb17c
I20260810 18:44:54.924952 24076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:44:54.939700 24058 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 18:44:54.955135 24076 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:44:54.956764 24076 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:44:54.969249 24076 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37: Generated new TSK 0
I20260810 18:44:54.970345 24076 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.228.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab.
Entry for principal kudu/127.4.228.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab.
I20260810 18:44:55.854583 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.228.212:34735/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.193:0
--local_ip_for_outbound_sockets=127.4.228.193
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:33847
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:32917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:44:56 dist-test-slave-7kzw krb5kdc[23257](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786387496, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 18:44:56.300280 24134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:44:56.300700 24134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:44:56.300915 24134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:44:56.312604 24134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:44:56.312753 24134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:44:56.312990 24134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.193
I20260810 18:44:56.328737 24134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:32917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.4.228.212:34735/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:33847
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.193
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:44:56.330705 24134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:44:56.332669 24134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:44:56.350555 24149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:44:56.351204 24150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:44:56.353322 24153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:44:56.353637 24134 server_base.cc:1034] running on GCE node
I20260810 18:44:56.354413 24134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:44:56.355645 24134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:44:56.356917 24134 hybrid_clock.cc:648] HybridClock initialized: now 1786387496356855 us; error 65 us; skew 500 ppm
I20260810 18:44:56.357327 24134 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:44:56 dist-test-slave-7kzw krb5kdc[23257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387496, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:44:56.361459 24134 init.cc:377] Logged in from keytab as kudu/127.4.228.193@KRBTEST.COM (short username kudu)
I20260810 18:44:56.363829 24134 webserver.cc:460] Webserver started at http://127.4.228.193:37307/ using document root <none> and password file <none>
I20260810 18:44:56.364621 24134 fs_manager.cc:359] Metadata directory not provided
I20260810 18:44:56.364785 24134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:44:56.365185 24134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 18:44:56 dist-test-slave-7kzw krb5kdc[23257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387496, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for HTTP/127.4.228.212@KRBTEST.COM
I20260810 18:44:56.385983 24134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-0/data/instance:
uuid: "1b061536576846e28c51503b5625b207"
format_stamp: "Formatted at 2026-08-10 18:44:56 on dist-test-slave-7kzw"
server_key: "2a1558d9ddfd0aff3221fd94295ab83f"
server_key_iv: "4b038c039b4f8ea822e378d39ca18ace"
server_key_version: "kuduclusterkey@0"
I20260810 18:44:56.387333 24134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance:
uuid: "1b061536576846e28c51503b5625b207"
format_stamp: "Formatted at 2026-08-10 18:44:56 on dist-test-slave-7kzw"
server_key: "2a1558d9ddfd0aff3221fd94295ab83f"
server_key_iv: "4b038c039b4f8ea822e378d39ca18ace"
server_key_version: "kuduclusterkey@0"
I20260810 18:44:56.414958 24134 fs_manager.cc:691] Time spent creating directory manager: real 0.027s user 0.010s sys 0.017s
I20260810 18:44:56.446316 24167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:44:56.448647 24134 fs_manager.cc:725] Time spent opening block manager: real 0.017s user 0.009s sys 0.006s
I20260810 18:44:56.448949 24134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-0/wal
uuid: "1b061536576846e28c51503b5625b207"
format_stamp: "Formatted at 2026-08-10 18:44:56 on dist-test-slave-7kzw"
server_key: "2a1558d9ddfd0aff3221fd94295ab83f"
server_key_iv: "4b038c039b4f8ea822e378d39ca18ace"
server_key_version: "kuduclusterkey@0"
I20260810 18:44:56.449337 24134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:44:56.528934 24134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:44:56.534696 24134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:44:56.535110 24134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:44:56.542596 24134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:44:56.551081 24134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:44:56.551278 24134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260810 18:44:56.551482 24134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:44:56.551606 24134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 18:44:56 dist-test-slave-7kzw krb5kdc[23257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387496, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for HTTP/127.4.228.212@KRBTEST.COM
I20260810 18:44:56.736449 24134 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.193:41729
I20260810 18:44:56.736478 24346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.193:41729 every 8 connection(s)
I20260810 18:44:56.738286 24134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
I20260810 18:44:56.745150 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 24134
I20260810 18:44:56.745376 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance
I20260810 18:44:56.746246 5011 external_mini_cluster.cc:1428] Setting key 003f72f3f7d720d5180bd7be03709215
Aug 10 18:44:56 dist-test-slave-7kzw krb5kdc[23257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387496, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:44:56.809748 24351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:44:56.742750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:34809 (local address 127.4.228.254:33847)
0810 18:44:56.778289 (+ 35539us) server_negotiation.cc:206] Beginning negotiation
0810 18:44:56.778296 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:44:56.784586 (+ 6290us) server_negotiation.cc:407] Connection header received
0810 18:44:56.784669 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:44:56.784674 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:44:56.784784 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:44:56.784895 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:44:56.790853 (+ 5958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:56.792053 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:56.794559 (+ 2506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:56.795266 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:56.795370 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:44:56.800290 (+ 4920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:44:56.800297 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:44:56.800303 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:44:56.800369 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:44:56.804233 (+ 3864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:44:56.805308 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:44:56.805314 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:44:56.805319 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:44:56.805480 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:44:56.806044 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:44:56.806049 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:44:56.806052 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:44:56.806433 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:44:56.806602 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 18:44:56.809011 (+ 2409us) server_negotiation.cc:299] Negotiation successful
0810 18:44:56.809306 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35350,"thread_start_us":35545,"threads_started":1}
WARNING: no policy specified for kudu/127.4.228.194@KRBTEST.COM; defaulting to no policy
I20260810 18:44:56.812638 24350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:44:56.742023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:33847 (local address 127.4.228.193:34809)
0810 18:44:56.781293 (+ 39270us) negotiation.cc:107] Waiting for socket to connect
0810 18:44:56.781368 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0810 18:44:56.784072 (+ 2704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:44:56.788166 (+ 4094us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:44:56.788179 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:44:56.789146 (+ 967us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:44:56.790047 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:44:56.790073 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:44:56.792295 (+ 2222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:44:56.792301 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:44:56.794353 (+ 2052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:44:56.794369 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:44:56.795771 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:44:56.795779 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:44:56.796080 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:44:56.796979 (+ 899us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:44:56.797012 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:44:56.799969 (+ 2957us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:44:56.804452 (+ 4483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:44:56.804464 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:44:56.804486 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:44:56.805130 (+ 644us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:44:56.805631 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:44:56.805636 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:44:56.805640 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:44:56.805909 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:44:56.806721 (+ 812us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:44:56.806731 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:44:56.807218 (+ 487us) client_negotiation.cc:769] Sending connection context
0810 18:44:56.810268 (+ 3050us) client_negotiation.cc:240] Negotiation successful
0810 18:44:56.811028 (+ 760us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38857,"thread_start_us":44646,"threads_started":1}
I20260810 18:44:56.814952 24348 heartbeater.cc:347] Connected to a master server at 127.4.228.254:33847
I20260810 18:44:56.815318 24348 heartbeater.cc:464] Registering TS with master...
I20260810 18:44:56.816224 24348 heartbeater.cc:511] Master 127.4.228.254:33847 requested a full tablet report, sending...
I20260810 18:44:56.819103 24004 ts_manager.cc:194] Registered new tserver with Master: 1b061536576846e28c51503b5625b207 (127.4.228.193:41729)
I20260810 18:44:56.821399 24004 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.193@KRBTEST.COM'} at 127.4.228.193:34809
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab.
Entry for principal kudu/127.4.228.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab.
W20260810 18:44:56.867134 5011 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.4.228.194: real 0.118s user 0.000s sys 0.024s
I20260810 18:44:56.868214 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.228.212:34735/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.194:0
--local_ip_for_outbound_sockets=127.4.228.194
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:33847
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:32917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:44:56 dist-test-slave-7kzw krb5kdc[23257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387496, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for HTTP/127.4.228.212@KRBTEST.COM
W20260810 18:44:57.342931 24354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:44:57.343345 24354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:44:57.343532 24354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:44:57.352675 24354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:44:57.352789 24354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:44:57.352903 24354 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.194
I20260810 18:44:57.362953 24354 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:32917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.4.228.212:34735/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:33847
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.194
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:44:57.364243 24354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:44:57.365541 24354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:44:57.379289 24378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:44:57.379263 24376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:44:57.379856 24380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:44:57.380568 24354 server_base.cc:1034] running on GCE node
I20260810 18:44:57.381242 24354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:44:57.382241 24354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:44:57.383476 24354 hybrid_clock.cc:648] HybridClock initialized: now 1786387497383400 us; error 63 us; skew 500 ppm
I20260810 18:44:57.383848 24354 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:44:57 dist-test-slave-7kzw krb5kdc[23257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387497, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:44:57.387162 24354 init.cc:377] Logged in from keytab as kudu/127.4.228.194@KRBTEST.COM (short username kudu)
I20260810 18:44:57.388758 24354 webserver.cc:460] Webserver started at http://127.4.228.194:44423/ using document root <none> and password file <none>
I20260810 18:44:57.389477 24354 fs_manager.cc:359] Metadata directory not provided
I20260810 18:44:57.389629 24354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:44:57.389964 24354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 18:44:57 dist-test-slave-7kzw krb5kdc[23257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387497, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for HTTP/127.4.228.212@KRBTEST.COM
I20260810 18:44:57.407395 24354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-1/data/instance:
uuid: "5752b39f09e94703870600b343135fbe"
format_stamp: "Formatted at 2026-08-10 18:44:57 on dist-test-slave-7kzw"
server_key: "5a7f22749d103524d49a61d1b1a08493"
server_key_iv: "9fd49c46df0b6722831598aff7b07480"
server_key_version: "kuduclusterkey@0"
I20260810 18:44:57.408586 24354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance:
uuid: "5752b39f09e94703870600b343135fbe"
format_stamp: "Formatted at 2026-08-10 18:44:57 on dist-test-slave-7kzw"
server_key: "5a7f22749d103524d49a61d1b1a08493"
server_key_iv: "9fd49c46df0b6722831598aff7b07480"
server_key_version: "kuduclusterkey@0"
I20260810 18:44:57.415758 24354 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260810 18:44:57.432751 24391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:44:57.434381 24354 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:44:57.434605 24354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-1/wal
uuid: "5752b39f09e94703870600b343135fbe"
format_stamp: "Formatted at 2026-08-10 18:44:57 on dist-test-slave-7kzw"
server_key: "5a7f22749d103524d49a61d1b1a08493"
server_key_iv: "9fd49c46df0b6722831598aff7b07480"
server_key_version: "kuduclusterkey@0"
I20260810 18:44:57.434821 24354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:44:57.447731 24354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:44:57.452054 24354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:44:57.452387 24354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:44:57.453478 24354 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:44:57.455137 24354 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:44:57.455262 24354 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:44:57.455399 24354 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:44:57.455464 24354 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:44:57.497409 24354 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.194:37071
I20260810 18:44:57.497412 24572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.194:37071 every 8 connection(s)
I20260810 18:44:57.498910 24354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
I20260810 18:44:57.502925 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 24354
I20260810 18:44:57.503129 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance
I20260810 18:44:57.503757 5011 external_mini_cluster.cc:1428] Setting key 7055085eb73a1f0efeb04bfb9b8aaeb9
Aug 10 18:44:57 dist-test-slave-7kzw krb5kdc[23257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387497, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:44:57.525586 24576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:44:57.501842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:45147 (local address 127.4.228.254:33847)
0810 18:44:57.502335 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0810 18:44:57.502340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:44:57.503975 (+ 1635us) server_negotiation.cc:407] Connection header received
0810 18:44:57.506125 (+ 2150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:44:57.506131 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:44:57.506230 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:44:57.506351 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:44:57.508460 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:57.509651 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:57.512374 (+ 2723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:57.513044 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:57.513175 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:44:57.517970 (+ 4795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:44:57.517980 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:44:57.517988 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:44:57.518039 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:44:57.521447 (+ 3408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:44:57.522577 (+ 1130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:44:57.522584 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:44:57.522588 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:44:57.522772 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:44:57.523390 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:44:57.523395 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:44:57.523399 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:44:57.523820 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:44:57.523995 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 18:44:57.524951 (+ 956us) server_negotiation.cc:299] Negotiation successful
0810 18:44:57.525198 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":266,"threads_started":1}
I20260810 18:44:57.527065 24577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:44:57.502449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:33847 (local address 127.4.228.194:45147)
0810 18:44:57.503440 (+ 991us) negotiation.cc:107] Waiting for socket to connect
0810 18:44:57.503504 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0810 18:44:57.505510 (+ 2006us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:44:57.506675 (+ 1165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:44:57.506688 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:44:57.507359 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:44:57.508261 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:44:57.508290 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:44:57.509845 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:44:57.509853 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:44:57.512202 (+ 2349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:44:57.512221 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:44:57.513310 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:44:57.513319 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:44:57.513647 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:44:57.514513 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:44:57.514551 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:44:57.517655 (+ 3104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:44:57.521635 (+ 3980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:44:57.521648 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:44:57.521670 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:44:57.522384 (+ 714us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:44:57.522905 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:44:57.522913 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:44:57.522919 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:44:57.523228 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:44:57.523973 (+ 745us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:44:57.523985 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:44:57.524472 (+ 487us) client_negotiation.cc:769] Sending connection context
0810 18:44:57.524807 (+ 335us) client_negotiation.cc:240] Negotiation successful
0810 18:44:57.525441 (+ 634us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"thread_start_us":394,"threads_started":1}
I20260810 18:44:57.528977 24573 heartbeater.cc:347] Connected to a master server at 127.4.228.254:33847
I20260810 18:44:57.529325 24573 heartbeater.cc:464] Registering TS with master...
I20260810 18:44:57.530339 24573 heartbeater.cc:511] Master 127.4.228.254:33847 requested a full tablet report, sending...
I20260810 18:44:57.532258 24004 ts_manager.cc:194] Registered new tserver with Master: 5752b39f09e94703870600b343135fbe (127.4.228.194:37071)
I20260810 18:44:57.533393 24004 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.194@KRBTEST.COM'} at 127.4.228.194:45147
WARNING: no policy specified for kudu/127.4.228.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab.
Entry for principal kudu/127.4.228.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab.
I20260810 18:44:57.572120 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.228.212:34735/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.195:0
--local_ip_for_outbound_sockets=127.4.228.195
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:33847
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:32917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:44:57.825484 24348 heartbeater.cc:503] Master 127.4.228.254:33847 was elected leader, sending a full tablet report...
W20260810 18:44:57.939172 24581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:44:57.939473 24581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:44:57.939658 24581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:44:57.947060 24581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:44:57.947171 24581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:44:57.947340 24581 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.195
I20260810 18:44:57.957566 24581 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:32917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.4.228.212:34735/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:33847
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.195
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:44:57.959052 24581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:44:57.960399 24581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:44:57.974148 24588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:44:57.974175 24594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:44:57.974100 24598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:44:57.975664 24581 server_base.cc:1034] running on GCE node
I20260810 18:44:57.976353 24581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:44:57.977459 24581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:44:57.978718 24581 hybrid_clock.cc:648] HybridClock initialized: now 1786387497978641 us; error 69 us; skew 500 ppm
I20260810 18:44:57.979156 24581 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:44:57 dist-test-slave-7kzw krb5kdc[23257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387497, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:44:57.987030 24581 init.cc:377] Logged in from keytab as kudu/127.4.228.195@KRBTEST.COM (short username kudu)
I20260810 18:44:57.988802 24581 webserver.cc:460] Webserver started at http://127.4.228.195:34419/ using document root <none> and password file <none>
I20260810 18:44:57.989589 24581 fs_manager.cc:359] Metadata directory not provided
I20260810 18:44:57.989710 24581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:44:57.990005 24581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 18:44:57 dist-test-slave-7kzw krb5kdc[23257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387497, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for HTTP/127.4.228.212@KRBTEST.COM
I20260810 18:44:58.006331 24581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-2/data/instance:
uuid: "0ecf396b87834eb8b35ab0889a2f1b5e"
format_stamp: "Formatted at 2026-08-10 18:44:58 on dist-test-slave-7kzw"
server_key: "c776d7b1c8a20190a794dcdd39d746d7"
server_key_iv: "a9c95b99fb35370de39e5c3bb63f52dc"
server_key_version: "kuduclusterkey@0"
I20260810 18:44:58.007330 24581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance:
uuid: "0ecf396b87834eb8b35ab0889a2f1b5e"
format_stamp: "Formatted at 2026-08-10 18:44:58 on dist-test-slave-7kzw"
server_key: "c776d7b1c8a20190a794dcdd39d746d7"
server_key_iv: "a9c95b99fb35370de39e5c3bb63f52dc"
server_key_version: "kuduclusterkey@0"
I20260810 18:44:58.013098 24581 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 18:44:58.029538 24611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:44:58.031181 24581 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:44:58.031383 24581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-2/wal
uuid: "0ecf396b87834eb8b35ab0889a2f1b5e"
format_stamp: "Formatted at 2026-08-10 18:44:58 on dist-test-slave-7kzw"
server_key: "c776d7b1c8a20190a794dcdd39d746d7"
server_key_iv: "a9c95b99fb35370de39e5c3bb63f52dc"
server_key_version: "kuduclusterkey@0"
I20260810 18:44:58.031625 24581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:44:58.065776 24581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:44:58.069675 24581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:44:58.069998 24581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:44:58.071166 24581 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:44:58.072599 24581 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:44:58.072676 24581 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:44:58.072858 24581 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:44:58.072914 24581 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:44:58.109431 24581 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.195:39577
I20260810 18:44:58.109472 24799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.195:39577 every 8 connection(s)
I20260810 18:44:58.110828 24581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
I20260810 18:44:58.114578 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 24581
I20260810 18:44:58.114790 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance
I20260810 18:44:58.115374 5011 external_mini_cluster.cc:1428] Setting key ed5cfd9be2882bba8dbef6f713fd6cfd
Aug 10 18:44:58 dist-test-slave-7kzw krb5kdc[23257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387497, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:44:58.131495 24803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:44:58.113509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:40069 (local address 127.4.228.254:33847)
0810 18:44:58.113948 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0810 18:44:58.113954 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:44:58.115151 (+ 1197us) server_negotiation.cc:407] Connection header received
0810 18:44:58.117250 (+ 2099us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:44:58.117256 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:44:58.117368 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:44:58.117489 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:44:58.119111 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:58.119849 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:58.121451 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:58.121884 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:58.122015 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:44:58.125479 (+ 3464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:44:58.125484 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:44:58.125488 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:44:58.125520 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:44:58.128154 (+ 2634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:44:58.128893 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:44:58.128899 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:44:58.128901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:44:58.129065 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:44:58.129630 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:44:58.129634 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:44:58.129635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:44:58.129987 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:44:58.130146 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 18:44:58.131056 (+ 910us) server_negotiation.cc:299] Negotiation successful
0810 18:44:58.131243 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":219,"threads_started":1}
I20260810 18:44:58.132447 24805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:44:58.114079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:33847 (local address 127.4.228.195:40069)
0810 18:44:58.114966 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0810 18:44:58.115012 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 18:44:58.116979 (+ 1967us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:44:58.117765 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:44:58.117774 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:44:58.118230 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:44:58.118940 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:44:58.118958 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:44:58.120015 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:44:58.120020 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:44:58.121306 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:44:58.121319 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:44:58.122133 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:44:58.122142 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:44:58.122373 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:44:58.123051 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:44:58.123073 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:44:58.125262 (+ 2189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:44:58.128314 (+ 3052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:44:58.128323 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:44:58.128338 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:44:58.128757 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:44:58.129216 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:44:58.129222 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:44:58.129226 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:44:58.129512 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:44:58.130134 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:44:58.130140 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:44:58.130589 (+ 449us) client_negotiation.cc:769] Sending connection context
0810 18:44:58.130909 (+ 320us) client_negotiation.cc:240] Negotiation successful
0810 18:44:58.131349 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":397,"threads_started":1}
I20260810 18:44:58.133839 24801 heartbeater.cc:347] Connected to a master server at 127.4.228.254:33847
I20260810 18:44:58.134204 24801 heartbeater.cc:464] Registering TS with master...
I20260810 18:44:58.135064 24801 heartbeater.cc:511] Master 127.4.228.254:33847 requested a full tablet report, sending...
I20260810 18:44:58.136701 24004 ts_manager.cc:194] Registered new tserver with Master: 0ecf396b87834eb8b35ab0889a2f1b5e (127.4.228.195:39577)
I20260810 18:44:58.137488 24004 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.195@KRBTEST.COM'} at 127.4.228.195:40069
I20260810 18:44:58.143571 5011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:44:58.145329 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 24134
Aug 10 18:44:58 dist-test-slave-7kzw krb5kdc[23257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387494, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.193@KRBTEST.COM
I20260810 18:44:58.162878 24808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:44:58.146442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33746 (local address 127.4.228.193:41729)
0810 18:44:58.147151 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0810 18:44:58.147159 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:44:58.147184 (+ 25us) server_negotiation.cc:407] Connection header received
0810 18:44:58.147249 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:44:58.147255 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:44:58.147535 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:44:58.147801 (+ 266us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:44:58.149342 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:58.150400 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:58.151991 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:58.152539 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:58.152635 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:44:58.157180 (+ 4545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:44:58.157190 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:44:58.157220 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:44:58.157256 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:44:58.160274 (+ 3018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:44:58.160962 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:44:58.160969 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:44:58.160971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:44:58.161125 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:44:58.161455 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:44:58.161458 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:44:58.161460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:44:58.161795 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:44:58.162023 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0810 18:44:58.162317 (+ 294us) server_negotiation.cc:299] Negotiation successful
0810 18:44:58.162585 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":350,"threads_started":1}
I20260810 18:44:58.164275 24340 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:44:58.248138 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 24134
I20260810 18:44:58.273926 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 24354
Aug 10 18:44:58 dist-test-slave-7kzw krb5kdc[23257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387494, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.194@KRBTEST.COM
I20260810 18:44:58.291510 24812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:44:58.275047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54424 (local address 127.4.228.194:37071)
0810 18:44:58.275716 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0810 18:44:58.275724 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:44:58.275757 (+ 33us) server_negotiation.cc:407] Connection header received
0810 18:44:58.275823 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:44:58.275829 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:44:58.276060 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:44:58.276204 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:44:58.277608 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:58.278918 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:58.280282 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:58.280856 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:58.280964 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:44:58.285437 (+ 4473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:44:58.285448 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:44:58.285465 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:44:58.285507 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:44:58.288588 (+ 3081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:44:58.289337 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:44:58.289345 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:44:58.289347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:44:58.289487 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:44:58.289886 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:44:58.289889 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:44:58.289891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:44:58.290248 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:44:58.290531 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0810 18:44:58.290898 (+ 367us) server_negotiation.cc:299] Negotiation successful
0810 18:44:58.291109 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":350,"threads_started":1}
I20260810 18:44:58.293076 24561 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:44:58.377182 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 24354
I20260810 18:44:58.404508 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 24581
Aug 10 18:44:58 dist-test-slave-7kzw krb5kdc[23257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387494, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.195@KRBTEST.COM
I20260810 18:44:58.421756 24818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:44:58.405613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50760 (local address 127.4.228.195:39577)
0810 18:44:58.406321 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0810 18:44:58.406329 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:44:58.406359 (+ 30us) server_negotiation.cc:407] Connection header received
0810 18:44:58.406425 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:44:58.406431 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:44:58.406722 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:44:58.406928 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:44:58.408470 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:58.409469 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:58.410921 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:44:58.411481 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:44:58.411623 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:44:58.416065 (+ 4442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:44:58.416077 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:44:58.416091 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:44:58.416124 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:44:58.418900 (+ 2776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:44:58.419596 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:44:58.419603 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:44:58.419605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:44:58.419744 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:44:58.420141 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:44:58.420147 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:44:58.420150 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:44:58.420494 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:44:58.420851 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0810 18:44:58.421123 (+ 272us) server_negotiation.cc:299] Negotiation successful
0810 18:44:58.421392 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":325,"threads_started":1}
I20260810 18:44:58.423152 24793 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:44:58.507833 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 24581
I20260810 18:44:58.532899 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 23947
I20260810 18:44:58.534395 24050 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:44:58.641911 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 23947
I20260810 18:44:58.675930 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:33847
--webserver_interface=127.4.228.254
--webserver_port=34417
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:32917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:33847
--ranger_config_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.228.212:34735/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.228.212:34735/kms with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:44:59.031279 24824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:44:59.031672 24824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:44:59.031836 24824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:44:59.039039 24824 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:44:59.039187 24824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:44:59.039306 24824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:44:59.039409 24824 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:44:59.039503 24824 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:44:59.049700 24824 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:32917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.228.212:34735/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:33847
--ranger_config_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:33847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=34417
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:44:59.051129 24824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:44:59.052534 24824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:44:59.064991 24833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:44:59.065028 24836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:44:59.065068 24841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:44:59.066121 24824 server_base.cc:1034] running on GCE node
I20260810 18:44:59.067065 24824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:44:59.068569 24824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:44:59.069841 24824 hybrid_clock.cc:648] HybridClock initialized: now 1786387499069786 us; error 61 us; skew 500 ppm
I20260810 18:44:59.070246 24824 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:44:59 dist-test-slave-7kzw krb5kdc[23257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387499, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:44:59.073737 24824 init.cc:377] Logged in from keytab as kudu/127.4.228.254@KRBTEST.COM (short username kudu)
I20260810 18:44:59.075358 24824 webserver.cc:460] Webserver started at http://127.4.228.254:34417/ using document root <none> and password file <none>
I20260810 18:44:59.076035 24824 fs_manager.cc:359] Metadata directory not provided
I20260810 18:44:59.076151 24824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 18:44:59 dist-test-slave-7kzw krb5kdc[23257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387499, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for HTTP/127.4.228.212@KRBTEST.COM
I20260810 18:44:59.247100 24824 fs_manager.cc:709] Time spent opening directory manager: real 0.021s user 0.018s sys 0.000s
I20260810 18:44:59.261803 24852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:44:59.268126 24824 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.007s sys 0.003s
I20260810 18:44:59.268889 24824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "e41cf7c93c134bccbda0e0982bf85b37"
format_stamp: "Formatted at 2026-08-10 18:44:54 on dist-test-slave-7kzw"
server_key: "ccc7a12e1bbca2dfa8d1a7c4167f57af"
server_key_iv: "608d5747c5aa86def6a3b7aa4d739b33"
server_key_version: "kuduclusterkey@0"
I20260810 18:44:59.277573 24824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:44:59.321363 24824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:44:59.341650 24824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:44:59.342577 24824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:44:59.423208 24824 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:33847
I20260810 18:44:59.423442 24949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:33847 every 8 connection(s)
I20260810 18:44:59.429638 24824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:44:59.442260 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 24824
I20260810 18:44:59.449965 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.228.212:34735/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.193:41729
--local_ip_for_outbound_sockets=127.4.228.193
--tserver_master_addrs=127.4.228.254:33847
--webserver_port=37307
--webserver_interface=127.4.228.193
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:32917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.228.212:34735/kms with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:44:59.491410 24951 sys_catalog.cc:263] Verifying existing consensus state
I20260810 18:44:59.503876 24951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37: Bootstrap starting.
I20260810 18:44:59.535591 24951 log.cc:824] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37: Log is configured to *not* fsync() on all Append() calls
I20260810 18:44:59.552330 24951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:44:59.553273 24951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37: Bootstrap complete.
I20260810 18:44:59.572811 24951 raft_consensus.cc:348] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41cf7c93c134bccbda0e0982bf85b37" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 33847 } }
I20260810 18:44:59.574539 24951 raft_consensus.cc:729] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e41cf7c93c134bccbda0e0982bf85b37, State: Initialized, Role: FOLLOWER
I20260810 18:44:59.575748 24951 consensus_queue.cc:260] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41cf7c93c134bccbda0e0982bf85b37" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 33847 } }
I20260810 18:44:59.576234 24951 raft_consensus.cc:388] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:44:59.576460 24951 raft_consensus.cc:482] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:44:59.576762 24951 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37 [term 1 FOLLOWER]: Advancing to term 2
I20260810 18:44:59.585232 24951 raft_consensus.cc:504] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41cf7c93c134bccbda0e0982bf85b37" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 33847 } }
I20260810 18:44:59.586231 24951 leader_election.cc:302] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e41cf7c93c134bccbda0e0982bf85b37; no voters:
I20260810 18:44:59.587605 24951 leader_election.cc:288] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 18:44:59.587843 24960 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 18:44:59.590204 24960 raft_consensus.cc:686] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37 [term 2 LEADER]: Becoming Leader. State: Replica: e41cf7c93c134bccbda0e0982bf85b37, State: Running, Role: LEADER
I20260810 18:44:59.591363 24960 consensus_queue.cc:237] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41cf7c93c134bccbda0e0982bf85b37" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 33847 } }
I20260810 18:44:59.592371 24951 sys_catalog.cc:564] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:44:59.596264 24961 sys_catalog.cc:455] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e41cf7c93c134bccbda0e0982bf85b37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41cf7c93c134bccbda0e0982bf85b37" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 33847 } } }
I20260810 18:44:59.597988 24961 sys_catalog.cc:458] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37 [sys.catalog]: This master's current role is: LEADER
I20260810 18:44:59.597414 24962 sys_catalog.cc:455] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e41cf7c93c134bccbda0e0982bf85b37. Latest consensus state: current_term: 2 leader_uuid: "e41cf7c93c134bccbda0e0982bf85b37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41cf7c93c134bccbda0e0982bf85b37" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 33847 } } }
I20260810 18:44:59.598734 24962 sys_catalog.cc:458] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37 [sys.catalog]: This master's current role is: LEADER
I20260810 18:44:59.612376 24970 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:44:59.626201 24970 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:44:59.630322 24970 catalog_manager.cc:1259] Loaded cluster ID: 7c481587ca70453b83351ce0322bb17c
I20260810 18:44:59.630599 24970 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:44:59.640578 24970 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:44:59.644119 24970 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37: Loaded TSK: 0
I20260810 18:44:59.648474 24970 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:44:59.656076 24951 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260810 18:45:00.312434 24957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:45:00.313284 24957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:45:00.313724 24957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:45:00.337512 24957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:45:00.337973 24957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:45:00.338485 24957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.193
I20260810 18:45:00.370563 24957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:32917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.228.212:34735/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.193:41729
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.228.193
--webserver_port=37307
--tserver_master_addrs=127.4.228.254:33847
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.193
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:45:00.374650 24957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:45:00.378434 24957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:45:00.443078 25016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:00.447249 25020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:00.448704 25014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:45:00.450486 24957 server_base.cc:1034] running on GCE node
I20260810 18:45:00.451704 24957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:45:00.459496 24957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:45:00.461735 24957 hybrid_clock.cc:648] HybridClock initialized: now 1786387500461590 us; error 142 us; skew 500 ppm
I20260810 18:45:00.462669 24957 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:45:00 dist-test-slave-7kzw krb5kdc[23257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387500, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:45:00.471663 24957 init.cc:377] Logged in from keytab as kudu/127.4.228.193@KRBTEST.COM (short username kudu)
I20260810 18:45:00.484823 24957 webserver.cc:460] Webserver started at http://127.4.228.193:37307/ using document root <none> and password file <none>
I20260810 18:45:00.490862 24957 fs_manager.cc:359] Metadata directory not provided
I20260810 18:45:00.491266 24957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 18:45:00 dist-test-slave-7kzw krb5kdc[23257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387500, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for HTTP/127.4.228.212@KRBTEST.COM
I20260810 18:45:00.603585 24957 fs_manager.cc:709] Time spent opening directory manager: real 0.030s user 0.013s sys 0.000s
I20260810 18:45:00.611722 25034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:00.615625 24957 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.003s sys 0.002s
I20260810 18:45:00.616010 24957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-0/wal
uuid: "1b061536576846e28c51503b5625b207"
format_stamp: "Formatted at 2026-08-10 18:44:56 on dist-test-slave-7kzw"
server_key: "2a1558d9ddfd0aff3221fd94295ab83f"
server_key_iv: "4b038c039b4f8ea822e378d39ca18ace"
server_key_version: "kuduclusterkey@0"
I20260810 18:45:00.619908 24957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:45:00.712949 24957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:45:00.723034 24957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:45:00.723912 24957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:45:00.727335 24957 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:45:00.731091 24957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:45:00.731334 24957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:00.731600 24957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:45:00.731794 24957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:00.991551 24957 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.193:41729
I20260810 18:45:00.995450 24957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
I20260810 18:45:00.997363 25218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.193:41729 every 8 connection(s)
I20260810 18:45:00.999650 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 24957
I20260810 18:45:01.000867 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.228.212:34735/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.194:37071
--local_ip_for_outbound_sockets=127.4.228.194
--tserver_master_addrs=127.4.228.254:33847
--webserver_port=44423
--webserver_interface=127.4.228.194
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:32917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.228.212:34735/kms with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:45:01 dist-test-slave-7kzw krb5kdc[23257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387500, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:45:01.091558 25227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:01.012941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:33847 (local address 127.4.228.193:57281)
0810 18:45:01.026518 (+ 13577us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:01.026635 (+ 117us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:01.034949 (+ 8314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:01.038411 (+ 3462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:01.038439 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:01.039703 (+ 1264us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:45:01.042207 (+ 2504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:01.042261 (+ 54us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:01.051733 (+ 9472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:01.051751 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:01.055607 (+ 3856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:01.055641 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:01.062042 (+ 6401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:01.062072 (+ 30us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:01.062683 (+ 611us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:01.065595 (+ 2912us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:45:01.065668 (+ 73us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:45:01.070967 (+ 5299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:45:01.080056 (+ 9089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:01.080070 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:01.080089 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:01.081617 (+ 1528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:01.082882 (+ 1265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:01.082895 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:01.082902 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:01.083381 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:01.085786 (+ 2405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:45:01.085797 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:45:01.086672 (+ 875us) client_negotiation.cc:769] Sending connection context
0810 18:45:01.087314 (+ 642us) client_negotiation.cc:240] Negotiation successful
0810 18:45:01.088498 (+ 1184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12938,"thread_start_us":12517,"threads_started":1}
I20260810 18:45:01.094686 25224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:01.015027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:57281 (local address 127.4.228.254:33847)
0810 18:45:01.027983 (+ 12956us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:01.028034 (+ 51us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:01.028206 (+ 172us) server_negotiation.cc:407] Connection header received
0810 18:45:01.036359 (+ 8153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:01.036395 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:01.037255 (+ 860us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:01.038088 (+ 833us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:01.042680 (+ 4592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:01.049013 (+ 6333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:01.058638 (+ 9625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:01.060857 (+ 2219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:01.061249 (+ 392us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:01.071595 (+ 10346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:01.071705 (+ 110us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:01.071754 (+ 49us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:01.071859 (+ 105us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:01.079655 (+ 7796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:01.082001 (+ 2346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:01.082028 (+ 27us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:01.082049 (+ 21us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:01.082555 (+ 506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:01.083730 (+ 1175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:01.083747 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:01.083753 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:01.085242 (+ 1489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:01.085608 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:01.090439 (+ 4831us) server_negotiation.cc:299] Negotiation successful
0810 18:45:01.091502 (+ 1063us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12484,"thread_start_us":12126,"threads_started":1}
I20260810 18:45:01.103768 25221 heartbeater.cc:347] Connected to a master server at 127.4.228.254:33847
I20260810 18:45:01.104466 25221 heartbeater.cc:464] Registering TS with master...
I20260810 18:45:01.106159 25221 heartbeater.cc:511] Master 127.4.228.254:33847 requested a full tablet report, sending...
W20260810 18:45:01.114045 25221 heartbeater.cc:644] Failed to heartbeat to 127.4.228.254:33847 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 18:45:01.115228 25221 heartbeater.cc:464] Registering TS with master...
I20260810 18:45:01.117125 25221 heartbeater.cc:511] Master 127.4.228.254:33847 requested a full tablet report, sending...
I20260810 18:45:01.120282 25221 heartbeater.cc:464] Registering TS with master...
I20260810 18:45:01.120981 25221 heartbeater.cc:511] Master 127.4.228.254:33847 requested a full tablet report, sending...
W20260810 18:45:01.123502 25221 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 18:45:01.667240 25228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:45:01.667725 25228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:45:01.667939 25228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:45:01.675635 25228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:45:01.675742 25228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:45:01.675854 25228 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.194
I20260810 18:45:01.686612 25228 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:32917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.228.212:34735/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.194:37071
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.228.194
--webserver_port=44423
--tserver_master_addrs=127.4.228.254:33847
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.194
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:45:01.688072 25228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:45:01.689487 25228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:45:01.706843 25244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:01.706807 25246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:01.707814 25248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:45:01.708392 25228 server_base.cc:1034] running on GCE node
I20260810 18:45:01.709165 25228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:45:01.710372 25228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:45:01.711649 25228 hybrid_clock.cc:648] HybridClock initialized: now 1786387501711573 us; error 60 us; skew 500 ppm
I20260810 18:45:01.712019 25228 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:45:01 dist-test-slave-7kzw krb5kdc[23257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387501, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:45:01.715617 25228 init.cc:377] Logged in from keytab as kudu/127.4.228.194@KRBTEST.COM (short username kudu)
I20260810 18:45:01.717425 25228 webserver.cc:460] Webserver started at http://127.4.228.194:44423/ using document root <none> and password file <none>
I20260810 18:45:01.718214 25228 fs_manager.cc:359] Metadata directory not provided
I20260810 18:45:01.718387 25228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 18:45:01 dist-test-slave-7kzw krb5kdc[23257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387501, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for HTTP/127.4.228.212@KRBTEST.COM
I20260810 18:45:01.743724 25228 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 18:45:01.747640 25267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:01.749668 25228 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 18:45:01.749888 25228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-1/wal
uuid: "5752b39f09e94703870600b343135fbe"
format_stamp: "Formatted at 2026-08-10 18:44:57 on dist-test-slave-7kzw"
server_key: "5a7f22749d103524d49a61d1b1a08493"
server_key_iv: "9fd49c46df0b6722831598aff7b07480"
server_key_version: "kuduclusterkey@0"
I20260810 18:45:01.751044 25228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:45:01.774132 25228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:45:01.778674 25228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:45:01.779052 25228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:45:01.780256 25228 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:45:01.781932 25228 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:45:01.782068 25228 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:01.782215 25228 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:45:01.782325 25228 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:01.889917 25228 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.194:37071
I20260810 18:45:01.890004 25449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.194:37071 every 8 connection(s)
I20260810 18:45:01.892033 25228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
I20260810 18:45:01.901780 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 25228
I20260810 18:45:01.902726 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.228.212:34735/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.195:39577
--local_ip_for_outbound_sockets=127.4.228.195
--tserver_master_addrs=127.4.228.254:33847
--webserver_port=34419
--webserver_interface=127.4.228.195
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:32917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.228.212:34735/kms with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:45:01 dist-test-slave-7kzw krb5kdc[23257](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786387501, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 10 18:45:01 dist-test-slave-7kzw krb5kdc[23257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387501, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:45:01.933737 25454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:01.896196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:36109 (local address 127.4.228.254:33847)
0810 18:45:01.900401 (+ 4205us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:01.900412 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:01.909451 (+ 9039us) server_negotiation.cc:407] Connection header received
0810 18:45:01.913601 (+ 4150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:01.913608 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:01.913720 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:01.913808 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:01.916331 (+ 2523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:01.917504 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:01.920065 (+ 2561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:01.920800 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:01.920905 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:01.925671 (+ 4766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:01.925679 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:01.925683 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:01.925729 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:01.929580 (+ 3851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:01.930637 (+ 1057us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:01.930643 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:01.930648 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:01.930839 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:01.931461 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:01.931466 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:01.931470 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:01.931834 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:01.931988 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:01.933064 (+ 1076us) server_negotiation.cc:299] Negotiation successful
0810 18:45:01.933341 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4006,"thread_start_us":3592,"threads_started":1}
I20260810 18:45:01.937209 25455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:01.896942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:33847 (local address 127.4.228.194:36109)
0810 18:45:01.909204 (+ 12262us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:01.909282 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:01.912880 (+ 3598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:01.914429 (+ 1549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:01.914443 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:01.915182 (+ 739us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:45:01.916092 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:01.916123 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:01.917741 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:01.917746 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:01.919861 (+ 2115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:01.919878 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:01.921318 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:01.921326 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:01.921629 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:01.922498 (+ 869us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:45:01.922543 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:45:01.925374 (+ 2831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:45:01.929763 (+ 4389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:01.929774 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:01.929795 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:01.930442 (+ 647us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:01.931007 (+ 565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:01.931012 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:01.931016 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:01.931318 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:01.932101 (+ 783us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:45:01.932110 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:45:01.932591 (+ 481us) client_negotiation.cc:769] Sending connection context
0810 18:45:01.934740 (+ 2149us) client_negotiation.cc:240] Negotiation successful
0810 18:45:01.935371 (+ 631us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11846,"thread_start_us":12379,"threads_started":1}
I20260810 18:45:01.939494 25451 heartbeater.cc:347] Connected to a master server at 127.4.228.254:33847
I20260810 18:45:01.939837 25451 heartbeater.cc:464] Registering TS with master...
I20260810 18:45:01.940796 25451 heartbeater.cc:511] Master 127.4.228.254:33847 requested a full tablet report, sending...
I20260810 18:45:01.943274 24883 ts_manager.cc:194] Registered new tserver with Master: 5752b39f09e94703870600b343135fbe (127.4.228.194:37071)
I20260810 18:45:01.945616 24883 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.194@KRBTEST.COM'} at 127.4.228.194:36109
I20260810 18:45:02.123644 25221 heartbeater.cc:464] Registering TS with master...
I20260810 18:45:02.124163 25221 heartbeater.cc:511] Master 127.4.228.254:33847 requested a full tablet report, sending...
I20260810 18:45:02.125674 24883 ts_manager.cc:194] Registered new tserver with Master: 1b061536576846e28c51503b5625b207 (127.4.228.193:41729)
I20260810 18:45:02.126914 24883 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.193@KRBTEST.COM'} at 127.4.228.193:57281
W20260810 18:45:02.358067 25456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:45:02.358644 25456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:45:02.358894 25456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:45:02.371415 25456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:45:02.371663 25456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:45:02.371929 25456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.195
I20260810 18:45:02.388157 25456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:32917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.228.212:34735/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.195:39577
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.228.195
--webserver_port=34419
--tserver_master_addrs=127.4.228.254:33847
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.195
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:45:02.390267 25456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:45:02.392238 25456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:45:02.444835 25468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:02.445082 25463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:02.445250 25464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:45:02.446730 25456 server_base.cc:1034] running on GCE node
I20260810 18:45:02.447962 25456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:45:02.449602 25456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:45:02.453814 25456 hybrid_clock.cc:648] HybridClock initialized: now 1786387502453737 us; error 39 us; skew 500 ppm
I20260810 18:45:02.454375 25456 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:45:02 dist-test-slave-7kzw krb5kdc[23257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387502, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:45:02.459089 25456 init.cc:377] Logged in from keytab as kudu/127.4.228.195@KRBTEST.COM (short username kudu)
I20260810 18:45:02.461197 25456 webserver.cc:460] Webserver started at http://127.4.228.195:34419/ using document root <none> and password file <none>
I20260810 18:45:02.462242 25456 fs_manager.cc:359] Metadata directory not provided
I20260810 18:45:02.462496 25456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 18:45:02 dist-test-slave-7kzw krb5kdc[23257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387501, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for HTTP/127.4.228.212@KRBTEST.COM
Aug 10 18:45:02 dist-test-slave-7kzw krb5kdc[23257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387502, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for HTTP/127.4.228.212@KRBTEST.COM
I20260810 18:45:02.503613 25456 fs_manager.cc:709] Time spent opening directory manager: real 0.020s user 0.010s sys 0.000s
I20260810 18:45:02.510751 25479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:02.512660 25456 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 18:45:02.512938 25456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-2/wal
uuid: "0ecf396b87834eb8b35ab0889a2f1b5e"
format_stamp: "Formatted at 2026-08-10 18:44:58 on dist-test-slave-7kzw"
server_key: "c776d7b1c8a20190a794dcdd39d746d7"
server_key_iv: "a9c95b99fb35370de39e5c3bb63f52dc"
server_key_version: "kuduclusterkey@0"
I20260810 18:45:02.514693 25456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:45:02.663314 25456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:45:02.668817 25456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:45:02.669185 25456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:45:02.670604 25456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:45:02.672485 25456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:45:02.672607 25456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:02.672734 25456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:45:02.672814 25456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:02.755970 25456 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.195:39577
I20260810 18:45:02.755980 25654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.195:39577 every 8 connection(s)
I20260810 18:45:02.758345 25456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
I20260810 18:45:02.764447 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 25456
Aug 10 18:45:02 dist-test-slave-7kzw krb5kdc[23257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387502, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:45:02.813658 25661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:02.766145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44130 (local address 127.4.228.254:33847)
0810 18:45:02.768151 (+ 2006us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:02.768159 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:02.770906 (+ 2747us) server_negotiation.cc:407] Connection header received
0810 18:45:02.771366 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:02.771373 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:02.771487 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:02.771594 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:02.780180 (+ 8586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:02.781322 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:02.792336 (+ 11014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:02.793124 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:02.793247 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:02.802671 (+ 9424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:02.802680 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:02.802686 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:02.802752 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:02.807206 (+ 4454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:02.808891 (+ 1685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:02.808897 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:02.808901 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:02.809073 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:02.810746 (+ 1673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:02.810752 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:02.810755 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:02.811169 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:02.811529 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:02.812986 (+ 1457us) server_negotiation.cc:299] Negotiation successful
0810 18:45:02.813252 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1865,"thread_start_us":2061,"threads_started":1}
I20260810 18:45:02.822571 25658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:02.762946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:34317 (local address 127.4.228.254:33847)
0810 18:45:02.763642 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:02.763652 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:02.766750 (+ 3098us) server_negotiation.cc:407] Connection header received
0810 18:45:02.775825 (+ 9075us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:02.775834 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:02.776004 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:02.776143 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:02.787825 (+ 11682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:02.789328 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:02.798106 (+ 8778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:02.798947 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:02.799092 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:02.814397 (+ 15305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:02.814405 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:02.814411 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:02.814494 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:02.818756 (+ 4262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:02.819699 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:02.819705 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:02.819717 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:02.819856 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:02.820378 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:02.820383 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:02.820387 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:02.820858 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:02.821014 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:02.821899 (+ 885us) server_negotiation.cc:299] Negotiation successful
0810 18:45:02.822126 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":3338,"threads_started":1}
I20260810 18:45:02.823829 25659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:02.764198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:33847 (local address 127.4.228.195:34317)
0810 18:45:02.765232 (+ 1034us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:02.765289 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:02.770442 (+ 5153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:02.782103 (+ 11661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:02.782120 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:02.782946 (+ 826us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:45:02.783878 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:02.783907 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:02.789591 (+ 5684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:02.789599 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:02.792114 (+ 2515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:02.792135 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:02.799606 (+ 7471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:02.799628 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:02.800092 (+ 464us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:02.801197 (+ 1105us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:45:02.801241 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:45:02.810319 (+ 9078us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:45:02.818929 (+ 8610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:02.818945 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:02.818961 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:02.819564 (+ 603us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:02.819997 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:02.820000 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:02.820003 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:02.820241 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:02.821008 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:45:02.821017 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:45:02.821408 (+ 391us) client_negotiation.cc:769] Sending connection context
0810 18:45:02.821781 (+ 373us) client_negotiation.cc:240] Negotiation successful
0810 18:45:02.822415 (+ 634us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":689,"thread_start_us":3944,"threads_started":1}
I20260810 18:45:02.825405 25655 heartbeater.cc:347] Connected to a master server at 127.4.228.254:33847
I20260810 18:45:02.825778 25655 heartbeater.cc:464] Registering TS with master...
I20260810 18:45:02.826635 25655 heartbeater.cc:511] Master 127.4.228.254:33847 requested a full tablet report, sending...
I20260810 18:45:02.828334 24883 ts_manager.cc:194] Registered new tserver with Master: 0ecf396b87834eb8b35ab0889a2f1b5e (127.4.228.195:39577)
I20260810 18:45:02.829221 24883 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.195@KRBTEST.COM'} at 127.4.228.195:34317
I20260810 18:45:02.832054 5011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:45:02.861228 25658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:02.843805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44138 (local address 127.4.228.254:33847)
0810 18:45:02.844023 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:02.844034 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:02.844566 (+ 532us) server_negotiation.cc:407] Connection header received
0810 18:45:02.844897 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:02.844903 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:02.844994 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:02.845071 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:02.847534 (+ 2463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:02.848392 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:02.849805 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:02.850270 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:02.850424 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:02.853809 (+ 3385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:02.853816 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:02.853822 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:02.853869 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:02.857460 (+ 3591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:02.858438 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:02.858444 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:02.858448 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:02.858646 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:02.859184 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:02.859192 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:02.859197 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:02.859604 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:02.859882 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:02.860567 (+ 685us) server_negotiation.cc:299] Negotiation successful
0810 18:45:02.860841 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 18:45:02.868685 24883 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44138:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:45:02.879387 24883 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
Aug 10 18:45:02 dist-test-slave-7kzw krb5kdc[23257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387501, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for HTTP/127.4.228.212@KRBTEST.COM
I20260810 18:45:02.951031 25451 heartbeater.cc:503] Master 127.4.228.254:33847 was elected leader, sending a full tablet report...
I20260810 18:45:02.955021 25678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:02.917800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50770 (local address 127.4.228.195:39577)
0810 18:45:02.936925 (+ 19125us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:02.936936 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:02.939661 (+ 2725us) server_negotiation.cc:407] Connection header received
0810 18:45:02.939841 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:02.939852 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:02.940096 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:02.940297 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:45:02.941357 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:02.943867 (+ 2510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:02.950685 (+ 6818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:02.952197 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:02.952300 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:02.952614 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:02.954234 (+ 1620us) server_negotiation.cc:299] Negotiation successful
0810 18:45:02.954515 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18895,"thread_start_us":18477,"threads_started":1}
I20260810 18:45:02.955238 25677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:02.913523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:41729 (local address 127.0.0.1:33748)
0810 18:45:02.938372 (+ 24849us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:02.938432 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:02.938803 (+ 371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:02.939915 (+ 1112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:02.939923 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:02.939954 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:45:02.940347 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:02.940359 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:02.949298 (+ 8939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:02.949304 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:02.951550 (+ 2246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:02.951569 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:02.954360 (+ 2791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:02.954367 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:02.954575 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:02.954609 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 18:45:02.954676 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 18:45:02.954885 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23328,"thread_start_us":26242,"threads_started":1}
I20260810 18:45:02.956238 25675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:02.917372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:39577 (local address 127.0.0.1:50770)
0810 18:45:02.939535 (+ 22163us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:02.939559 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:02.939680 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:02.940647 (+ 967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:02.940651 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:02.940678 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:45:02.941111 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:02.941125 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:02.944129 (+ 3004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:02.944138 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:02.949496 (+ 5358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:02.949519 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:02.952887 (+ 3368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:02.952896 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:02.953187 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:02.953260 (+ 73us) client_negotiation.cc:769] Sending connection context
0810 18:45:02.955589 (+ 2329us) client_negotiation.cc:240] Negotiation successful
0810 18:45:02.955779 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21994,"thread_start_us":19796,"threads_started":1}
I20260810 18:45:02.958213 25676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:02.914291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33748 (local address 127.4.228.193:41729)
0810 18:45:02.938957 (+ 24666us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:02.938966 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:02.939000 (+ 34us) server_negotiation.cc:407] Connection header received
0810 18:45:02.939116 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:02.939124 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:02.939397 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:02.939608 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:45:02.945819 (+ 6211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:02.947551 (+ 1732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:02.952308 (+ 4757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:02.953753 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:02.956349 (+ 2596us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:02.957578 (+ 1229us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:02.957665 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 18:45:02.957854 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24479,"thread_start_us":26167,"threads_started":1}
I20260810 18:45:02.961890 25553 tablet_service.cc:1467] Processing CreateTablet for tablet 7f0d4fadfc974ca8a17f7b1e49c28da4 (DEFAULT_TABLE table=test-table [id=cf2043a5853a4504a8884d806af48356]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:45:02.964145 25113 tablet_service.cc:1467] Processing CreateTablet for tablet 7f0d4fadfc974ca8a17f7b1e49c28da4 (DEFAULT_TABLE table=test-table [id=cf2043a5853a4504a8884d806af48356]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:45:02.965610 25553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f0d4fadfc974ca8a17f7b1e49c28da4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:45:02.966678 25113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f0d4fadfc974ca8a17f7b1e49c28da4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:45:02.972710 25677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:02.946646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:37071 (local address 127.0.0.1:54428)
0810 18:45:02.955687 (+ 9041us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:02.955707 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:02.955828 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:02.962172 (+ 6344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:02.962177 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:02.962215 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:45:02.962393 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:02.962405 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:02.965971 (+ 3566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:02.965977 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:02.967853 (+ 1876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:02.967869 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:02.971874 (+ 4005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:02.971880 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:02.972126 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:02.972170 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 18:45:02.972252 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 18:45:02.972405 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8931,"thread_start_us":12761,"threads_started":1}
I20260810 18:45:02.991194 25683 tablet_bootstrap.cc:492] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 0ecf396b87834eb8b35ab0889a2f1b5e: Bootstrap starting.
I20260810 18:45:02.993551 25679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:02.940628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54428 (local address 127.4.228.194:37071)
0810 18:45:02.961487 (+ 20859us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:02.961496 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:02.961530 (+ 34us) server_negotiation.cc:407] Connection header received
0810 18:45:02.961646 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:02.961654 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:02.961947 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:02.962161 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:45:02.962689 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:02.965790 (+ 3101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:02.968811 (+ 3021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:02.971671 (+ 2860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:02.992155 (+ 20484us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:02.992461 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:02.992960 (+ 499us) server_negotiation.cc:299] Negotiation successful
0810 18:45:02.993206 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20683,"thread_start_us":21716,"threads_started":1}
I20260810 18:45:02.997931 25353 tablet_service.cc:1467] Processing CreateTablet for tablet 7f0d4fadfc974ca8a17f7b1e49c28da4 (DEFAULT_TABLE table=test-table [id=cf2043a5853a4504a8884d806af48356]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:45:03.000064 25353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f0d4fadfc974ca8a17f7b1e49c28da4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:45:03.014580 25684 tablet_bootstrap.cc:492] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 1b061536576846e28c51503b5625b207: Bootstrap starting.
I20260810 18:45:03.018738 25684 tablet_bootstrap.cc:654] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 1b061536576846e28c51503b5625b207: Neither blocks nor log segments found. Creating new log.
I20260810 18:45:03.019646 25683 tablet_bootstrap.cc:654] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 0ecf396b87834eb8b35ab0889a2f1b5e: Neither blocks nor log segments found. Creating new log.
I20260810 18:45:03.021227 25683 log.cc:824] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 0ecf396b87834eb8b35ab0889a2f1b5e: Log is configured to *not* fsync() on all Append() calls
I20260810 18:45:03.026846 25684 log.cc:824] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 1b061536576846e28c51503b5625b207: Log is configured to *not* fsync() on all Append() calls
I20260810 18:45:03.037720 25684 tablet_bootstrap.cc:492] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 1b061536576846e28c51503b5625b207: No bootstrap required, opened a new log
I20260810 18:45:03.041203 25683 tablet_bootstrap.cc:492] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 0ecf396b87834eb8b35ab0889a2f1b5e: No bootstrap required, opened a new log
I20260810 18:45:03.041687 25683 ts_tablet_manager.cc:1397] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 0ecf396b87834eb8b35ab0889a2f1b5e: Time spent bootstrapping tablet: real 0.051s user 0.011s sys 0.000s
I20260810 18:45:03.053361 25684 ts_tablet_manager.cc:1397] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 1b061536576846e28c51503b5625b207: Time spent bootstrapping tablet: real 0.039s user 0.004s sys 0.008s
I20260810 18:45:03.059377 25689 tablet_bootstrap.cc:492] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 5752b39f09e94703870600b343135fbe: Bootstrap starting.
I20260810 18:45:03.068661 25684 raft_consensus.cc:348] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 1b061536576846e28c51503b5625b207 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b061536576846e28c51503b5625b207" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41729 } } peers { permanent_uuid: "5752b39f09e94703870600b343135fbe" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 37071 } } peers { permanent_uuid: "0ecf396b87834eb8b35ab0889a2f1b5e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 39577 } }
I20260810 18:45:03.073657 25689 tablet_bootstrap.cc:654] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 5752b39f09e94703870600b343135fbe: Neither blocks nor log segments found. Creating new log.
I20260810 18:45:03.077714 25683 raft_consensus.cc:348] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 0ecf396b87834eb8b35ab0889a2f1b5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b061536576846e28c51503b5625b207" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41729 } } peers { permanent_uuid: "5752b39f09e94703870600b343135fbe" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 37071 } } peers { permanent_uuid: "0ecf396b87834eb8b35ab0889a2f1b5e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 39577 } }
I20260810 18:45:03.079911 25689 log.cc:824] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 5752b39f09e94703870600b343135fbe: Log is configured to *not* fsync() on all Append() calls
I20260810 18:45:03.080228 25683 raft_consensus.cc:374] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 0ecf396b87834eb8b35ab0889a2f1b5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:45:03.080951 25683 raft_consensus.cc:729] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 0ecf396b87834eb8b35ab0889a2f1b5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ecf396b87834eb8b35ab0889a2f1b5e, State: Initialized, Role: FOLLOWER
I20260810 18:45:03.082021 25683 consensus_queue.cc:260] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 0ecf396b87834eb8b35ab0889a2f1b5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b061536576846e28c51503b5625b207" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41729 } } peers { permanent_uuid: "5752b39f09e94703870600b343135fbe" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 37071 } } peers { permanent_uuid: "0ecf396b87834eb8b35ab0889a2f1b5e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 39577 } }
I20260810 18:45:03.083601 25684 raft_consensus.cc:374] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 1b061536576846e28c51503b5625b207 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:45:03.084264 25684 raft_consensus.cc:729] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 1b061536576846e28c51503b5625b207 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b061536576846e28c51503b5625b207, State: Initialized, Role: FOLLOWER
I20260810 18:45:03.086033 25684 consensus_queue.cc:260] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 1b061536576846e28c51503b5625b207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b061536576846e28c51503b5625b207" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41729 } } peers { permanent_uuid: "5752b39f09e94703870600b343135fbe" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 37071 } } peers { permanent_uuid: "0ecf396b87834eb8b35ab0889a2f1b5e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 39577 } }
I20260810 18:45:03.092022 25655 heartbeater.cc:503] Master 127.4.228.254:33847 was elected leader, sending a full tablet report...
I20260810 18:45:03.095041 25683 ts_tablet_manager.cc:1428] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 0ecf396b87834eb8b35ab0889a2f1b5e: Time spent starting tablet: real 0.053s user 0.008s sys 0.010s
I20260810 18:45:03.098687 25221 heartbeater.cc:503] Master 127.4.228.254:33847 was elected leader, sending a full tablet report...
I20260810 18:45:03.102274 25689 tablet_bootstrap.cc:492] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 5752b39f09e94703870600b343135fbe: No bootstrap required, opened a new log
I20260810 18:45:03.102777 25689 ts_tablet_manager.cc:1397] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 5752b39f09e94703870600b343135fbe: Time spent bootstrapping tablet: real 0.044s user 0.014s sys 0.000s
I20260810 18:45:03.108141 25684 ts_tablet_manager.cc:1428] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 1b061536576846e28c51503b5625b207: Time spent starting tablet: real 0.055s user 0.016s sys 0.000s
I20260810 18:45:03.112723 25689 raft_consensus.cc:348] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 5752b39f09e94703870600b343135fbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b061536576846e28c51503b5625b207" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41729 } } peers { permanent_uuid: "5752b39f09e94703870600b343135fbe" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 37071 } } peers { permanent_uuid: "0ecf396b87834eb8b35ab0889a2f1b5e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 39577 } }
I20260810 18:45:03.113356 25689 raft_consensus.cc:374] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 5752b39f09e94703870600b343135fbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:45:03.113484 25689 raft_consensus.cc:729] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 5752b39f09e94703870600b343135fbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5752b39f09e94703870600b343135fbe, State: Initialized, Role: FOLLOWER
I20260810 18:45:03.114342 25689 consensus_queue.cc:260] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 5752b39f09e94703870600b343135fbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b061536576846e28c51503b5625b207" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41729 } } peers { permanent_uuid: "5752b39f09e94703870600b343135fbe" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 37071 } } peers { permanent_uuid: "0ecf396b87834eb8b35ab0889a2f1b5e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 39577 } }
I20260810 18:45:03.117496 25689 ts_tablet_manager.cc:1428] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 5752b39f09e94703870600b343135fbe: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
W20260810 18:45:03.150811 25452 tablet.cc:2369] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 5752b39f09e94703870600b343135fbe: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:45:03.257098 25222 tablet.cc:2369] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 1b061536576846e28c51503b5625b207: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:45:03.285741 25656 tablet.cc:2369] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 0ecf396b87834eb8b35ab0889a2f1b5e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:45:03.371140 25694 raft_consensus.cc:482] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 0ecf396b87834eb8b35ab0889a2f1b5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:45:03.371476 25694 raft_consensus.cc:504] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 0ecf396b87834eb8b35ab0889a2f1b5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b061536576846e28c51503b5625b207" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41729 } } peers { permanent_uuid: "5752b39f09e94703870600b343135fbe" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 37071 } } peers { permanent_uuid: "0ecf396b87834eb8b35ab0889a2f1b5e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 39577 } }
I20260810 18:45:03.373225 25694 leader_election.cc:288] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 0ecf396b87834eb8b35ab0889a2f1b5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b061536576846e28c51503b5625b207 (127.4.228.193:41729), 5752b39f09e94703870600b343135fbe (127.4.228.194:37071)
I20260810 18:45:03.380388 25676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:03.373616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:60465 (local address 127.4.228.193:41729)
0810 18:45:03.373876 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:03.373883 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:03.374161 (+ 278us) server_negotiation.cc:407] Connection header received
0810 18:45:03.374535 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:03.374544 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:03.374655 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:03.374738 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:45:03.375657 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:03.376763 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:03.378698 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:03.379479 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:03.379662 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:03.379743 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:03.380009 (+ 266us) server_negotiation.cc:299] Negotiation successful
0810 18:45:03.380137 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260810 18:45:03.380432 25717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:03.373445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:41729 (local address 127.4.228.195:60465)
0810 18:45:03.374037 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:03.374060 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:03.374286 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:03.374999 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:03.375005 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:03.375039 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:45:03.375391 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:03.375406 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:03.376929 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:03.376934 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:03.378523 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:03.378544 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:03.379650 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:03.379656 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:03.379845 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:03.379870 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 18:45:03.379978 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 18:45:03.380105 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":322,"threads_started":1}
I20260810 18:45:03.381994 25679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:03.374487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:46359 (local address 127.4.228.194:37071)
0810 18:45:03.374691 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:03.374697 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:03.375371 (+ 674us) server_negotiation.cc:407] Connection header received
0810 18:45:03.375565 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:03.375571 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:03.375669 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:03.375772 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:45:03.376320 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:03.377152 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:03.379694 (+ 2542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:03.380884 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:03.381034 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:03.381103 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:03.381621 (+ 518us) server_negotiation.cc:299] Negotiation successful
0810 18:45:03.381756 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 18:45:03.381776 25149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f0d4fadfc974ca8a17f7b1e49c28da4" candidate_uuid: "0ecf396b87834eb8b35ab0889a2f1b5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b061536576846e28c51503b5625b207" is_pre_election: true
I20260810 18:45:03.382023 25718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:03.374228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:37071 (local address 127.4.228.195:46359)
0810 18:45:03.375226 (+ 998us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:03.375244 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:03.375398 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:03.375787 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:03.375791 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:03.375814 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:45:03.376141 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:03.376150 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:03.377307 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:03.377313 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:03.379489 (+ 2176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:03.379504 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:03.381255 (+ 1751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:03.381262 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:03.381437 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:03.381468 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 18:45:03.381565 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 18:45:03.381698 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":858,"thread_start_us":635,"threads_started":1}
I20260810 18:45:03.382439 25149 raft_consensus.cc:2454] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 1b061536576846e28c51503b5625b207 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ecf396b87834eb8b35ab0889a2f1b5e in term 0.
I20260810 18:45:03.383263 25484 leader_election.cc:302] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 0ecf396b87834eb8b35ab0889a2f1b5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ecf396b87834eb8b35ab0889a2f1b5e, 1b061536576846e28c51503b5625b207; no voters:
I20260810 18:45:03.383399 25381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f0d4fadfc974ca8a17f7b1e49c28da4" candidate_uuid: "0ecf396b87834eb8b35ab0889a2f1b5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5752b39f09e94703870600b343135fbe" is_pre_election: true
I20260810 18:45:03.383744 25694 raft_consensus.cc:2781] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 0ecf396b87834eb8b35ab0889a2f1b5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:45:03.383891 25694 raft_consensus.cc:482] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 0ecf396b87834eb8b35ab0889a2f1b5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:45:03.383975 25694 raft_consensus.cc:3037] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 0ecf396b87834eb8b35ab0889a2f1b5e [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:45:03.383924 25381 raft_consensus.cc:2454] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 5752b39f09e94703870600b343135fbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ecf396b87834eb8b35ab0889a2f1b5e in term 0.
I20260810 18:45:03.387197 25694 raft_consensus.cc:504] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 0ecf396b87834eb8b35ab0889a2f1b5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b061536576846e28c51503b5625b207" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41729 } } peers { permanent_uuid: "5752b39f09e94703870600b343135fbe" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 37071 } } peers { permanent_uuid: "0ecf396b87834eb8b35ab0889a2f1b5e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 39577 } }
I20260810 18:45:03.387924 25694 leader_election.cc:288] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 0ecf396b87834eb8b35ab0889a2f1b5e [CANDIDATE]: Term 1 election: Requested vote from peers 1b061536576846e28c51503b5625b207 (127.4.228.193:41729), 5752b39f09e94703870600b343135fbe (127.4.228.194:37071)
I20260810 18:45:03.388499 25149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f0d4fadfc974ca8a17f7b1e49c28da4" candidate_uuid: "0ecf396b87834eb8b35ab0889a2f1b5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b061536576846e28c51503b5625b207"
I20260810 18:45:03.388574 25381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f0d4fadfc974ca8a17f7b1e49c28da4" candidate_uuid: "0ecf396b87834eb8b35ab0889a2f1b5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5752b39f09e94703870600b343135fbe"
I20260810 18:45:03.388751 25149 raft_consensus.cc:3037] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 1b061536576846e28c51503b5625b207 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:45:03.388814 25381 raft_consensus.cc:3037] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 5752b39f09e94703870600b343135fbe [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:45:03.392092 25149 raft_consensus.cc:2454] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 1b061536576846e28c51503b5625b207 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ecf396b87834eb8b35ab0889a2f1b5e in term 1.
I20260810 18:45:03.392091 25381 raft_consensus.cc:2454] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 5752b39f09e94703870600b343135fbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ecf396b87834eb8b35ab0889a2f1b5e in term 1.
I20260810 18:45:03.392534 25484 leader_election.cc:302] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 0ecf396b87834eb8b35ab0889a2f1b5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ecf396b87834eb8b35ab0889a2f1b5e, 1b061536576846e28c51503b5625b207; no voters:
I20260810 18:45:03.392899 25694 raft_consensus.cc:2781] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 0ecf396b87834eb8b35ab0889a2f1b5e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:45:03.393476 25694 raft_consensus.cc:686] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 0ecf396b87834eb8b35ab0889a2f1b5e [term 1 LEADER]: Becoming Leader. State: Replica: 0ecf396b87834eb8b35ab0889a2f1b5e, State: Running, Role: LEADER
I20260810 18:45:03.394142 25694 consensus_queue.cc:237] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 0ecf396b87834eb8b35ab0889a2f1b5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b061536576846e28c51503b5625b207" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41729 } } peers { permanent_uuid: "5752b39f09e94703870600b343135fbe" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 37071 } } peers { permanent_uuid: "0ecf396b87834eb8b35ab0889a2f1b5e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 39577 } }
I20260810 18:45:03.399827 24881 catalog_manager.cc:5495] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 0ecf396b87834eb8b35ab0889a2f1b5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ecf396b87834eb8b35ab0889a2f1b5e (127.4.228.195). New cstate: current_term: 1 leader_uuid: "0ecf396b87834eb8b35ab0889a2f1b5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b061536576846e28c51503b5625b207" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5752b39f09e94703870600b343135fbe" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 37071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ecf396b87834eb8b35ab0889a2f1b5e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 39577 } health_report { overall_health: HEALTHY } } }
I20260810 18:45:03.439345 25678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:03.430333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50134 (local address 127.4.228.195:39577)
0810 18:45:03.430586 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:03.430593 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:03.433503 (+ 2910us) server_negotiation.cc:407] Connection header received
0810 18:45:03.433772 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:03.433776 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:03.433860 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:03.434025 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:45:03.434445 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:03.435330 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:03.437005 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:03.437415 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:03.437554 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:03.438085 (+ 531us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:45:03.438698 (+ 613us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:45:03.438861 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:03.438946 (+ 85us) server_negotiation.cc:299] Negotiation successful
0810 18:45:03.439028 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 18:45:03.448666 25149 raft_consensus.cc:1261] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 1b061536576846e28c51503b5625b207 [term 1 FOLLOWER]: Refusing update from remote peer 0ecf396b87834eb8b35ab0889a2f1b5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:45:03.448657 25381 raft_consensus.cc:1261] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 5752b39f09e94703870600b343135fbe [term 1 FOLLOWER]: Refusing update from remote peer 0ecf396b87834eb8b35ab0889a2f1b5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:45:03.449855 25719 consensus_queue.cc:1035] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 0ecf396b87834eb8b35ab0889a2f1b5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5752b39f09e94703870600b343135fbe" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 37071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:45:03.450273 25694 consensus_queue.cc:1035] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 0ecf396b87834eb8b35ab0889a2f1b5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b061536576846e28c51503b5625b207" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 41729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:45:03.495512 24883 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:45:03.495838 24883 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:45:03.500250 24883 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e41cf7c93c134bccbda0e0982bf85b37: Sending DeleteTablet for 3 replicas of tablet 7f0d4fadfc974ca8a17f7b1e49c28da4
I20260810 18:45:03.501983 25553 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f0d4fadfc974ca8a17f7b1e49c28da4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:45:03 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:50770
I20260810 18:45:03.502274 25113 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f0d4fadfc974ca8a17f7b1e49c28da4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:45:03 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:33748
I20260810 18:45:03.503036 25353 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f0d4fadfc974ca8a17f7b1e49c28da4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:45:03 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:54428
I20260810 18:45:03.507953 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 24957
I20260810 18:45:03.537644 25739 tablet_replica.cc:330] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 5752b39f09e94703870600b343135fbe: stopping tablet replica
I20260810 18:45:03.538316 25739 raft_consensus.cc:2229] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 5752b39f09e94703870600b343135fbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:45:03.539129 25739 raft_consensus.cc:2258] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 5752b39f09e94703870600b343135fbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:45:03.547165 25738 tablet_replica.cc:330] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 1b061536576846e28c51503b5625b207: stopping tablet replica
I20260810 18:45:03.547818 25738 raft_consensus.cc:2229] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 1b061536576846e28c51503b5625b207 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:45:03.548488 25738 raft_consensus.cc:2258] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 1b061536576846e28c51503b5625b207 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:45:03.551322 25739 ts_tablet_manager.cc:1905] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 5752b39f09e94703870600b343135fbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:45:03.551627 25738 ts_tablet_manager.cc:1905] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 1b061536576846e28c51503b5625b207: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:45:03.556581 25737 tablet_replica.cc:330] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 0ecf396b87834eb8b35ab0889a2f1b5e: stopping tablet replica
I20260810 18:45:03.558218 25737 raft_consensus.cc:2229] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 0ecf396b87834eb8b35ab0889a2f1b5e [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:45:03.559051 25737 raft_consensus.cc:2258] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 0ecf396b87834eb8b35ab0889a2f1b5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:45:03.559167 25739 ts_tablet_manager.cc:1918] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 5752b39f09e94703870600b343135fbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:45:03.559338 25739 log.cc:1196] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 5752b39f09e94703870600b343135fbe: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-1/wal/wals/7f0d4fadfc974ca8a17f7b1e49c28da4
I20260810 18:45:03.559813 25739 ts_tablet_manager.cc:1939] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 5752b39f09e94703870600b343135fbe: Deleting consensus metadata
I20260810 18:45:03.561311 25737 ts_tablet_manager.cc:1905] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 0ecf396b87834eb8b35ab0889a2f1b5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:45:03.561625 24859 catalog_manager.cc:4841] TS 5752b39f09e94703870600b343135fbe (127.4.228.194:37071): tablet 7f0d4fadfc974ca8a17f7b1e49c28da4 (table test-table [id=cf2043a5853a4504a8884d806af48356]) successfully deleted
I20260810 18:45:03.565179 25738 ts_tablet_manager.cc:1918] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 1b061536576846e28c51503b5625b207: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:45:03.565342 25738 log.cc:1196] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 1b061536576846e28c51503b5625b207: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-0/wal/wals/7f0d4fadfc974ca8a17f7b1e49c28da4
I20260810 18:45:03.565855 25738 ts_tablet_manager.cc:1939] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 1b061536576846e28c51503b5625b207: Deleting consensus metadata
I20260810 18:45:03.568323 24859 catalog_manager.cc:4841] TS 1b061536576846e28c51503b5625b207 (127.4.228.193:41729): tablet 7f0d4fadfc974ca8a17f7b1e49c28da4 (table test-table [id=cf2043a5853a4504a8884d806af48356]) successfully deleted
I20260810 18:45:03.574922 25676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:03.509218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47304 (local address 127.4.228.193:41729)
0810 18:45:03.509400 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:03.509407 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:03.540191 (+ 30784us) server_negotiation.cc:407] Connection header received
0810 18:45:03.540553 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:03.540560 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:03.540658 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:03.540741 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:03.542977 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:03.544328 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:03.549107 (+ 4779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:03.549755 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:03.549986 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:03.566938 (+ 16952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:03.566959 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:03.566981 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:03.567031 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:03.571239 (+ 4208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:03.572257 (+ 1018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:03.572267 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:03.572272 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:03.572444 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:03.572932 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:03.572937 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:03.572940 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:03.573410 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:03.573759 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:03.574052 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 18:45:03.574281 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 18:45:03.575598 25214 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:45:03.583880 25737 ts_tablet_manager.cc:1918] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 0ecf396b87834eb8b35ab0889a2f1b5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:45:03.584091 25737 log.cc:1196] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 0ecf396b87834eb8b35ab0889a2f1b5e: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387267874959-5011-0/minicluster-data/ts-2/wal/wals/7f0d4fadfc974ca8a17f7b1e49c28da4
I20260810 18:45:03.584698 25737 ts_tablet_manager.cc:1939] T 7f0d4fadfc974ca8a17f7b1e49c28da4 P 0ecf396b87834eb8b35ab0889a2f1b5e: Deleting consensus metadata
I20260810 18:45:03.586654 24853 catalog_manager.cc:4841] TS 0ecf396b87834eb8b35ab0889a2f1b5e (127.4.228.195:39577): tablet 7f0d4fadfc974ca8a17f7b1e49c28da4 (table test-table [id=cf2043a5853a4504a8884d806af48356]) successfully deleted
I20260810 18:45:03.718638 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 24957
I20260810 18:45:03.754130 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 25228
I20260810 18:45:03.774497 25679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:03.755893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39834 (local address 127.4.228.194:37071)
0810 18:45:03.756107 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:03.756113 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:03.756134 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:45:03.756202 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:03.756208 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:03.756289 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:03.756397 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:03.758368 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:03.759505 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:03.761518 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:03.762158 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:03.762284 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:03.765946 (+ 3662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:03.765965 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:03.765985 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:03.766039 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:03.770137 (+ 4098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:03.771181 (+ 1044us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:03.771191 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:03.771195 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:03.771382 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:03.771861 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:03.771866 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:03.771870 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:03.772344 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:03.772649 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:03.773883 (+ 1234us) server_negotiation.cc:299] Negotiation successful
0810 18:45:03.774096 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 18:45:03.775466 25443 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:45:03.876645 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 25228
I20260810 18:45:03.914363 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 25456
I20260810 18:45:03.933931 25678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:03.915581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50148 (local address 127.4.228.195:39577)
0810 18:45:03.915770 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:03.915774 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:03.915796 (+ 22us) server_negotiation.cc:407] Connection header received
0810 18:45:03.915982 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:03.915987 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:03.916065 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:03.916176 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:03.918281 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:03.919360 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:03.921552 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:03.922032 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:03.922155 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:03.926425 (+ 4270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:03.926442 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:03.926516 (+ 74us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:03.926565 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:03.930209 (+ 3644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:03.931359 (+ 1150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:03.931368 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:03.931371 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:03.931569 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:03.932123 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:03.932127 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:03.932130 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:03.932558 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:03.932827 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:03.933331 (+ 504us) server_negotiation.cc:299] Negotiation successful
0810 18:45:03.933531 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 18:45:03.934914 25647 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:45:04.062585 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 25456
I20260810 18:45:04.093443 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 24824
I20260810 18:45:04.094929 24936 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:45:04.210769 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 24824
I20260810 18:45:04.249514 5011 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 18:45:04.595515 5011 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 18:45:04.595844 5011 mini_ranger.cc:66] Stopping Ranger...
I20260810 18:45:04.942075 5011 mini_ranger.cc:68] Stopped Ranger
2026-08-10 18:45:04.942 UTC [23305] LOG: received smart shutdown request
2026-08-10 18:45:04.946 UTC [23305] LOG: background worker "logical replication launcher" (PID 23317) exited with exit code 1
2026-08-10 18:45:04.946 UTC [23312] LOG: shutting down
2026-08-10 18:45:05.118 UTC [23305] LOG: database system is shut down
2026-08-10T18:45:05Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (58247 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:45:05 dist-test-slave-7kzw krb5kdc[25764](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:45:05 dist-test-slave-7kzw krb5kdc[25764](info): set up 2 sockets
Aug 10 18:45:05 dist-test-slave-7kzw krb5kdc[25764](info): commencing operation
krb5kdc: starting...
W20260810 18:45:06.584090 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.288s user 0.007s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:45:06 dist-test-slave-7kzw krb5kdc[25764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387506, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:45:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:45:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.228.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
I20260810 18:45:06.890295 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:42575
--webserver_interface=127.4.228.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:35185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:42575,127.4.228.253:43743,127.4.228.252:33825
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:45:07.244158 25783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:45:07.244475 25783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:45:07.244645 25783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:45:07.252034 25783 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:45:07.252154 25783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:45:07.252246 25783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:45:07.252355 25783 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:45:07.252421 25783 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:45:07.262732 25783 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:35185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:42575,127.4.228.253:43743,127.4.228.252:33825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:42575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:45:07.264158 25783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:45:07.265545 25783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:45:07.278080 25791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:07.278067 25797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:07.277989 25792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:45:07.279754 25783 server_base.cc:1034] running on GCE node
I20260810 18:45:07.280606 25783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:45:07.282147 25783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:45:07.283363 25783 hybrid_clock.cc:648] HybridClock initialized: now 1786387507283296 us; error 77 us; skew 500 ppm
I20260810 18:45:07.283774 25783 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:45:07 dist-test-slave-7kzw krb5kdc[25764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387507, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:45:07.287145 25783 init.cc:377] Logged in from keytab as kudu/127.4.228.254@KRBTEST.COM (short username kudu)
I20260810 18:45:07.288833 25783 webserver.cc:460] Webserver started at http://127.4.228.254:40857/ using document root <none> and password file <none>
I20260810 18:45:07.289592 25783 fs_manager.cc:359] Metadata directory not provided
I20260810 18:45:07.289747 25783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:45:07.290144 25783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:45:07.293324 25783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "38ff4aca9d0e4c37ae4ff101fb9d294b"
format_stamp: "Formatted at 2026-08-10 18:45:07 on dist-test-slave-7kzw"
server_key: "c864f78bc0ff203da94322e8bf1844d1"
server_key_iv: "00fb2fd7db36ae65280b7e161635b540"
server_key_version: "encryptionkey@0"
I20260810 18:45:07.294153 25783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "38ff4aca9d0e4c37ae4ff101fb9d294b"
format_stamp: "Formatted at 2026-08-10 18:45:07 on dist-test-slave-7kzw"
server_key: "c864f78bc0ff203da94322e8bf1844d1"
server_key_iv: "00fb2fd7db36ae65280b7e161635b540"
server_key_version: "encryptionkey@0"
I20260810 18:45:07.299690 25783 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 18:45:07.303416 25804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:07.304965 25783 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:45:07.305203 25783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "38ff4aca9d0e4c37ae4ff101fb9d294b"
format_stamp: "Formatted at 2026-08-10 18:45:07 on dist-test-slave-7kzw"
server_key: "c864f78bc0ff203da94322e8bf1844d1"
server_key_iv: "00fb2fd7db36ae65280b7e161635b540"
server_key_version: "encryptionkey@0"
I20260810 18:45:07.305431 25783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:45:07.319603 25783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:45:07.324397 25783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:45:07.324723 25783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:45:07.345124 25783 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:42575
I20260810 18:45:07.345180 25871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:42575 every 8 connection(s)
I20260810 18:45:07.346612 25783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:45:07.350322 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 25783
I20260810 18:45:07.350629 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:45:07.351051 25872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:45:07.351460 5011 external_mini_cluster.cc:1428] Setting key e24edda1ead50a17836908c295326efb
I20260810 18:45:07.358438 25872 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42575 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:45:07 dist-test-slave-7kzw krb5kdc[25764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387507, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.228.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.253@KRBTEST.COM" created.
I20260810 18:45:07.383299 25877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:07.361949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:42575 (local address 127.0.0.1:35782)
0810 18:45:07.363580 (+ 1631us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:07.363617 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:07.365549 (+ 1932us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:07.366449 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:07.366456 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:07.367281 (+ 825us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:45:07.367665 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:07.367684 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:07.369416 (+ 1732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:07.369422 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:07.370667 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:07.370681 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:07.371705 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:07.371713 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:07.372046 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:07.372806 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:45:07.372832 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:45:07.375451 (+ 2619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:45:07.378978 (+ 3527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:07.378992 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:07.379005 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:07.379443 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:07.379872 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:07.379880 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:07.379883 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:07.380042 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:07.380907 (+ 865us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:45:07.380920 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:45:07.381263 (+ 343us) client_negotiation.cc:769] Sending connection context
0810 18:45:07.381500 (+ 237us) client_negotiation.cc:240] Negotiation successful
0810 18:45:07.381956 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1423,"mutex_wait_us":75,"thread_start_us":481,"threads_started":1}
I20260810 18:45:07.383328 25878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:07.361949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35782 (local address 127.4.228.254:42575)
0810 18:45:07.363349 (+ 1400us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:07.363360 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:07.363759 (+ 399us) server_negotiation.cc:407] Connection header received
0810 18:45:07.365954 (+ 2195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:07.365960 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:07.366200 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:07.366424 (+ 224us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:07.367896 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:07.369231 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:07.370819 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:07.371409 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:07.371641 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:07.375703 (+ 4062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:07.375712 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:07.375725 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:07.375754 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:07.378848 (+ 3094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:07.379591 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:07.379597 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:07.379599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:07.379744 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:07.380184 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:07.380188 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:07.380190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:07.380700 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:07.380917 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:07.381782 (+ 865us) server_negotiation.cc:299] Negotiation successful
0810 18:45:07.382139 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":112,"server-negotiator.queue_time_us":1189,"spinlock_wait_cycles":87936,"thread_start_us":872,"threads_started":1}
I20260810 18:45:07.386534 25872 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.228.253" port: 43743 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:45:07.387864 25877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:07.387389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:43743 (local address 127.0.0.1:42294)
0810 18:45:07.387617 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:07.387746 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.253:43743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260810 18:45:07.388216 25808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.228.253:43743: connect: Connection refused (error 111)
I20260810 18:45:07.389964 25877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:07.389589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:43743 (local address 127.0.0.1:42298)
0810 18:45:07.389756 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:07.389833 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.253:43743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260810 18:45:07.390188 25872 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.253:43743: Network error: Client connection negotiation failed: client connection to 127.4.228.253:43743: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.253.keytab.
Entry for principal kudu/127.4.228.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.253.keytab.
I20260810 18:45:07.413869 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.253:43743
--webserver_interface=127.4.228.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.253.keytab
--principal=kudu/127.4.228.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:35185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:42575,127.4.228.253:43743,127.4.228.252:33825
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:45:07.439612 25872 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.253" port: 43743 } attempt: 1
I20260810 18:45:07.441044 25877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:07.440428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:43743 (local address 127.0.0.1:42308)
0810 18:45:07.440704 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:07.440865 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.253:43743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260810 18:45:07.442129 25877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:07.441735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:43743 (local address 127.0.0.1:42314)
0810 18:45:07.441889 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:07.441972 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.253:43743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 18:45:07.442391 25872 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.253:43743: Network error: Client connection negotiation failed: client connection to 127.4.228.253:43743: connect: Connection refused (error 111)
I20260810 18:45:07.510738 25872 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.253" port: 43743 } attempt: 2
I20260810 18:45:07.512108 25877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:07.511467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:43743 (local address 127.0.0.1:42326)
0810 18:45:07.511789 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:07.511912 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.253:43743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260810 18:45:07.513371 25877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:07.512901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:43743 (local address 127.0.0.1:42330)
0810 18:45:07.513074 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:07.513196 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.253:43743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260810 18:45:07.513664 25872 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.253:43743: Network error: Client connection negotiation failed: client connection to 127.4.228.253:43743: connect: Connection refused (error 111)
I20260810 18:45:07.604946 25872 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.253" port: 43743 } attempt: 3
I20260810 18:45:07.606318 25877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:07.605678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:43743 (local address 127.0.0.1:42334)
0810 18:45:07.605956 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:07.606102 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.253:43743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260810 18:45:07.607743 25877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:07.607153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:43743 (local address 127.0.0.1:42342)
0810 18:45:07.607387 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:07.607527 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.253:43743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260810 18:45:07.608107 25872 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.253:43743: Network error: Client connection negotiation failed: client connection to 127.4.228.253:43743: connect: Connection refused (error 111)
I20260810 18:45:07.751400 25872 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.253" port: 43743 } attempt: 4
I20260810 18:45:07.752907 25877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:07.752092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:43743 (local address 127.0.0.1:42346)
0810 18:45:07.752491 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:07.752610 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.253:43743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216}
I20260810 18:45:07.754168 25877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:07.753677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:43743 (local address 127.0.0.1:42348)
0810 18:45:07.753864 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:07.753997 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.253:43743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260810 18:45:07.754539 25872 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.253:43743: Network error: Client connection negotiation failed: client connection to 127.4.228.253:43743: connect: Connection refused (error 111)
W20260810 18:45:07.769629 25883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:45:07.769927 25883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:45:07.770133 25883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:45:07.777545 25883 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:45:07.777659 25883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:45:07.777748 25883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:45:07.777854 25883 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:45:07.777940 25883 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:45:07.788223 25883 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:35185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:42575,127.4.228.253:43743,127.4.228.252:33825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.253.keytab
--principal=kudu/127.4.228.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.253:43743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.228.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:45:07.789724 25883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:45:07.791240 25883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:45:07.804625 25891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:07.804690 25893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:07.804929 25895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:45:07.805745 25883 server_base.cc:1034] running on GCE node
I20260810 18:45:07.806694 25883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:45:07.808202 25883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:45:07.809449 25883 hybrid_clock.cc:648] HybridClock initialized: now 1786387507809403 us; error 34 us; skew 500 ppm
I20260810 18:45:07.809902 25883 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:45:07 dist-test-slave-7kzw krb5kdc[25764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387507, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:45:07.813427 25883 init.cc:377] Logged in from keytab as kudu/127.4.228.253@KRBTEST.COM (short username kudu)
I20260810 18:45:07.815049 25883 webserver.cc:460] Webserver started at http://127.4.228.253:39631/ using document root <none> and password file <none>
I20260810 18:45:07.815788 25883 fs_manager.cc:359] Metadata directory not provided
I20260810 18:45:07.815943 25883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:45:07.816274 25883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:45:07.819428 25883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-1/data/instance:
uuid: "977723500379436084d4571a618805dd"
format_stamp: "Formatted at 2026-08-10 18:45:07 on dist-test-slave-7kzw"
server_key: "fec68819aea15bbb45a6db9e69864ecc"
server_key_iv: "d1c0c5f21b428de0ae683b0df63396ab"
server_key_version: "encryptionkey@0"
I20260810 18:45:07.820243 25883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-1/wal/instance:
uuid: "977723500379436084d4571a618805dd"
format_stamp: "Formatted at 2026-08-10 18:45:07 on dist-test-slave-7kzw"
server_key: "fec68819aea15bbb45a6db9e69864ecc"
server_key_iv: "d1c0c5f21b428de0ae683b0df63396ab"
server_key_version: "encryptionkey@0"
I20260810 18:45:07.825515 25883 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 18:45:07.829201 25904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:07.830776 25883 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:45:07.830972 25883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-1/wal
uuid: "977723500379436084d4571a618805dd"
format_stamp: "Formatted at 2026-08-10 18:45:07 on dist-test-slave-7kzw"
server_key: "fec68819aea15bbb45a6db9e69864ecc"
server_key_iv: "d1c0c5f21b428de0ae683b0df63396ab"
server_key_version: "encryptionkey@0"
I20260810 18:45:07.831188 25883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:45:07.849148 25883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:45:07.853761 25883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:45:07.854069 25883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:45:07.874243 25883 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.253:43743
I20260810 18:45:07.874284 25988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.253:43743 every 8 connection(s)
I20260810 18:45:07.875731 25883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-1/data/info.pb
I20260810 18:45:07.879701 25989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:45:07.882144 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 25883
I20260810 18:45:07.882395 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-1/wal/instance
I20260810 18:45:07.883035 5011 external_mini_cluster.cc:1428] Setting key d4eca233848b71916f8cf1b443ac64e6
I20260810 18:45:07.886715 25989 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42575 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:45:07 dist-test-slave-7kzw krb5kdc[25764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387507, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.253@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:45:07.912271 25991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:07.889843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35786 (local address 127.4.228.254:42575)
0810 18:45:07.890409 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:07.890415 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:07.891569 (+ 1154us) server_negotiation.cc:407] Connection header received
0810 18:45:07.893627 (+ 2058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:07.893634 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:07.893729 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:07.893862 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:07.895765 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:07.896660 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:07.899286 (+ 2626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:07.899727 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:07.899844 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:07.904946 (+ 5102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:07.904952 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:07.904956 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:07.905006 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:07.907977 (+ 2971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:07.909190 (+ 1213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:07.909197 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:07.909202 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:07.909373 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:07.909982 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:07.909987 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:07.909989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:07.910523 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:07.910715 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:07.911815 (+ 1100us) server_negotiation.cc:299] Negotiation successful
0810 18:45:07.911993 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":327,"threads_started":1}
I20260810 18:45:07.913928 25993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:07.890415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:42575 (local address 127.0.0.1:35786)
0810 18:45:07.891377 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:07.891417 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:07.893316 (+ 1899us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:07.894123 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:07.894132 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:07.894755 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:45:07.895526 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:07.895553 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:07.896843 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:07.896851 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:07.899095 (+ 2244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:07.899121 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:07.899997 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:07.900006 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:07.900336 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:07.901192 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:45:07.901229 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:45:07.904603 (+ 3374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:45:07.908153 (+ 3550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:07.908167 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:07.908189 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:07.909004 (+ 815us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:07.909511 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:07.909520 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:07.909525 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:07.909800 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:07.910734 (+ 934us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:45:07.910746 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:45:07.911310 (+ 564us) client_negotiation.cc:769] Sending connection context
0810 18:45:07.911670 (+ 360us) client_negotiation.cc:240] Negotiation successful
0810 18:45:07.912275 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"thread_start_us":397,"threads_started":1}
WARNING: no policy specified for kudu/127.4.228.252@KRBTEST.COM; defaulting to no policy
I20260810 18:45:07.915881 25989 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.228.253" port: 43743 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.252@KRBTEST.COM" created.
Aug 10 18:45:07 dist-test-slave-7kzw krb5kdc[25764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387507, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.253@KRBTEST.COM for kudu/127.4.228.253@KRBTEST.COM
I20260810 18:45:07.932375 25993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:07.916753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:43743 (local address 127.0.0.1:42350)
0810 18:45:07.917082 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:07.917105 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:07.917230 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:07.918319 (+ 1089us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:07.918323 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:07.918825 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:45:07.918952 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:07.918961 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:07.920866 (+ 1905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:07.920873 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:07.921877 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:07.921892 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:07.922948 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:07.922954 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:07.923228 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:07.923920 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:45:07.923939 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:45:07.925935 (+ 1996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:45:07.929768 (+ 3833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:07.929773 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:07.929777 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:07.930133 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:07.930570 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:07.930576 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:07.930580 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:07.930723 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:07.931366 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:45:07.931371 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:45:07.931513 (+ 142us) client_negotiation.cc:769] Sending connection context
0810 18:45:07.931656 (+ 143us) client_negotiation.cc:240] Negotiation successful
0810 18:45:07.931944 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151}
I20260810 18:45:07.932484 25996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:07.917100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42350 (local address 127.4.228.253:43743)
0810 18:45:07.917900 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:07.917906 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:07.917933 (+ 27us) server_negotiation.cc:407] Connection header received
0810 18:45:07.917976 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:07.917982 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:07.918164 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:07.918326 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:07.919118 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:07.920671 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:07.922050 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:07.922771 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:07.922942 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:07.926152 (+ 3210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:07.926163 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:07.926178 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:07.926207 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:07.929619 (+ 3412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:07.930283 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:07.930293 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:07.930297 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:07.930421 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:07.930833 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:07.930840 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:07.930843 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:07.931196 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:07.931426 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:07.931804 (+ 378us) server_negotiation.cc:299] Negotiation successful
0810 18:45:07.932067 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":624,"thread_start_us":469,"threads_started":1}
I20260810 18:45:07.935794 25989 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.228.252" port: 33825 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:45:07.936951 25993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:07.936485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:33825 (local address 127.0.0.1:44360)
0810 18:45:07.936662 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:07.936784 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:33825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260810 18:45:07.937208 25906 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.228.252:33825: connect: Connection refused (error 111)
I20260810 18:45:07.938887 25993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:07.938537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:33825 (local address 127.0.0.1:44372)
0810 18:45:07.938687 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:07.938774 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:33825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 18:45:07.939134 25989 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.252:33825: Network error: Client connection negotiation failed: client connection to 127.4.228.252:33825: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.252.keytab.
Entry for principal kudu/127.4.228.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.252.keytab.
I20260810 18:45:07.954794 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.252:33825
--webserver_interface=127.4.228.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.252.keytab
--principal=kudu/127.4.228.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:35185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:42575,127.4.228.253:43743,127.4.228.252:33825
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:45:07.988461 25989 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.252" port: 33825 } attempt: 1
I20260810 18:45:07.989949 25993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:07.989310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:33825 (local address 127.0.0.1:44380)
0810 18:45:07.989563 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:07.989724 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:33825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260810 18:45:07.991168 25993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:07.990710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:33825 (local address 127.0.0.1:44394)
0810 18:45:07.990885 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:07.991009 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:33825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 18:45:07.991410 25989 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.252:33825: Network error: Client connection negotiation failed: client connection to 127.4.228.252:33825: connect: Connection refused (error 111)
I20260810 18:45:08.053869 25872 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.253" port: 43743 } attempt: 5
I20260810 18:45:08.059659 25989 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.252" port: 33825 } attempt: 2
I20260810 18:45:08.060825 25993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:08.060266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:33825 (local address 127.0.0.1:44402)
0810 18:45:08.060493 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:08.060613 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:33825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260810 18:45:08.062091 25993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:08.061600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:33825 (local address 127.0.0.1:44412)
0810 18:45:08.061803 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:08.061910 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:33825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260810 18:45:08.062413 25989 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.252:33825: Network error: Client connection negotiation failed: client connection to 127.4.228.252:33825: connect: Connection refused (error 111)
Aug 10 18:45:08 dist-test-slave-7kzw krb5kdc[25764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387507, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for kudu/127.4.228.253@KRBTEST.COM
I20260810 18:45:08.069983 25877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:08.054727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:43743 (local address 127.0.0.1:42352)
0810 18:45:08.054980 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:08.055003 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:08.055151 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:08.055628 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:08.055633 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:08.056218 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:45:08.056479 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:08.056497 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:08.057654 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:08.057660 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:08.059107 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:08.059125 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:08.060071 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:08.060081 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:08.060275 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:08.061031 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:45:08.061057 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:45:08.063554 (+ 2497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:45:08.067451 (+ 3897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:08.067455 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:08.067457 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:08.067866 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:08.068312 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:08.068316 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:08.068318 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:08.068466 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:08.069126 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:45:08.069130 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:45:08.069325 (+ 195us) client_negotiation.cc:769] Sending connection context
0810 18:45:08.069463 (+ 138us) client_negotiation.cc:240] Negotiation successful
0810 18:45:08.069727 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93}
I20260810 18:45:08.070150 25996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:08.055087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42352 (local address 127.4.228.253:43743)
0810 18:45:08.055278 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:08.055282 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:08.055302 (+ 20us) server_negotiation.cc:407] Connection header received
0810 18:45:08.055381 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:08.055385 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:08.055450 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:08.055538 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:08.056734 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:08.057492 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:08.059304 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:08.059834 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:08.060655 (+ 821us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:08.063765 (+ 3110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:08.063773 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:08.063778 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:08.063818 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:08.067238 (+ 3420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:08.067998 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:08.068004 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:08.068008 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:08.068168 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:08.068589 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:08.068594 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:08.068598 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:08.068978 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:08.069136 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:08.069625 (+ 489us) server_negotiation.cc:299] Negotiation successful
0810 18:45:08.069866 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 18:45:08.071300 25872 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.228.252" port: 33825 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:45:08.072335 25877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:08.071960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:33825 (local address 127.0.0.1:44418)
0810 18:45:08.072124 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:08.072208 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:33825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260810 18:45:08.073257 25877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:08.072918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:33825 (local address 127.0.0.1:44432)
0810 18:45:08.073063 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:08.073151 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:33825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260810 18:45:08.073524 25872 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.252:33825: Network error: Client connection negotiation failed: client connection to 127.4.228.252:33825: connect: Connection refused (error 111)
I20260810 18:45:08.124789 25872 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.252" port: 33825 } attempt: 1
I20260810 18:45:08.126067 25877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:08.125541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:33825 (local address 127.0.0.1:44448)
0810 18:45:08.125772 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:08.125886 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:33825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260810 18:45:08.127234 25877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:08.126845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:33825 (local address 127.0.0.1:44456)
0810 18:45:08.127005 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:08.127104 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:33825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 18:45:08.127538 25872 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.252:33825: Network error: Client connection negotiation failed: client connection to 127.4.228.252:33825: connect: Connection refused (error 111)
I20260810 18:45:08.153694 25989 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.252" port: 33825 } attempt: 3
I20260810 18:45:08.155072 25993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:08.154506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:33825 (local address 127.0.0.1:44464)
0810 18:45:08.154757 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:08.154860 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:33825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260810 18:45:08.156371 25993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:08.155883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:33825 (local address 127.0.0.1:44480)
0810 18:45:08.156166 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:08.156254 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:33825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175}
W20260810 18:45:08.156672 25989 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.252:33825: Network error: Client connection negotiation failed: client connection to 127.4.228.252:33825: connect: Connection refused (error 111)
I20260810 18:45:08.195820 25872 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.252" port: 33825 } attempt: 2
I20260810 18:45:08.197336 25877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:08.196754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:33825 (local address 127.0.0.1:44496)
0810 18:45:08.197037 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:08.197136 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:33825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260810 18:45:08.198568 25877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:08.198112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:33825 (local address 127.0.0.1:44508)
0810 18:45:08.198287 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:08.198403 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:33825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260810 18:45:08.198818 25872 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.252:33825: Network error: Client connection negotiation failed: client connection to 127.4.228.252:33825: connect: Connection refused (error 111)
I20260810 18:45:08.299924 25989 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.252" port: 33825 } attempt: 4
I20260810 18:45:08.301355 25993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:08.300793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:33825 (local address 127.0.0.1:44510)
0810 18:45:08.301066 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:08.301192 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:33825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260810 18:45:08.302441 25993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:08.302089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:33825 (local address 127.0.0.1:44526)
0810 18:45:08.302250 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:08.302332 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:33825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260810 18:45:08.302742 25989 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.252:33825: Network error: Client connection negotiation failed: client connection to 127.4.228.252:33825: connect: Connection refused (error 111)
I20260810 18:45:08.305045 25872 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.252" port: 33825 } attempt: 3
I20260810 18:45:08.306113 25877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:08.305679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:33825 (local address 127.0.0.1:44530)
0810 18:45:08.305872 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:08.305965 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:33825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260810 18:45:08.307315 25877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:08.306816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:33825 (local address 127.0.0.1:44538)
0810 18:45:08.307032 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:08.307115 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:33825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
W20260810 18:45:08.307520 25872 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.252:33825: Network error: Client connection negotiation failed: client connection to 127.4.228.252:33825: connect: Connection refused (error 111)
W20260810 18:45:08.315518 25999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:45:08.315846 25999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:45:08.316004 25999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:45:08.323616 25999 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:45:08.323767 25999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:45:08.323874 25999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:45:08.323979 25999 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:45:08.324055 25999 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:45:08.334156 25999 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:35185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:42575,127.4.228.253:43743,127.4.228.252:33825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.252.keytab
--principal=kudu/127.4.228.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.252:33825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.228.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:45:08.335808 25999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:45:08.337270 25999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:45:08.349905 26010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:08.349916 26009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:08.350203 26013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:45:08.353775 25999 server_base.cc:1034] running on GCE node
I20260810 18:45:08.354655 25999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:45:08.356166 25999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:45:08.357403 25999 hybrid_clock.cc:648] HybridClock initialized: now 1786387508357349 us; error 65 us; skew 500 ppm
I20260810 18:45:08.357769 25999 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:45:08 dist-test-slave-7kzw krb5kdc[25764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387508, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:45:08.360921 25999 init.cc:377] Logged in from keytab as kudu/127.4.228.252@KRBTEST.COM (short username kudu)
I20260810 18:45:08.362541 25999 webserver.cc:460] Webserver started at http://127.4.228.252:33703/ using document root <none> and password file <none>
I20260810 18:45:08.363272 25999 fs_manager.cc:359] Metadata directory not provided
I20260810 18:45:08.363427 25999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:45:08.363754 25999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:45:08.366861 25999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-2/data/instance:
uuid: "d8c3c67531434504afa24f3db2f87b4f"
format_stamp: "Formatted at 2026-08-10 18:45:08 on dist-test-slave-7kzw"
server_key: "93f467cdec6f5b800d14aeed53370b2a"
server_key_iv: "b13e3046c98a14945f9b4e482c6b96b5"
server_key_version: "encryptionkey@0"
I20260810 18:45:08.367707 25999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-2/wal/instance:
uuid: "d8c3c67531434504afa24f3db2f87b4f"
format_stamp: "Formatted at 2026-08-10 18:45:08 on dist-test-slave-7kzw"
server_key: "93f467cdec6f5b800d14aeed53370b2a"
server_key_iv: "b13e3046c98a14945f9b4e482c6b96b5"
server_key_version: "encryptionkey@0"
I20260810 18:45:08.373178 25999 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 18:45:08.376874 26023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:08.378253 25999 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260810 18:45:08.378445 25999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-2/wal
uuid: "d8c3c67531434504afa24f3db2f87b4f"
format_stamp: "Formatted at 2026-08-10 18:45:08 on dist-test-slave-7kzw"
server_key: "93f467cdec6f5b800d14aeed53370b2a"
server_key_iv: "b13e3046c98a14945f9b4e482c6b96b5"
server_key_version: "encryptionkey@0"
I20260810 18:45:08.378676 25999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:45:08.400274 25999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:45:08.404760 25999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:45:08.405071 25999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:45:08.425161 25999 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.252:33825
I20260810 18:45:08.425218 26101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.252:33825 every 8 connection(s)
I20260810 18:45:08.426620 25999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-2/data/info.pb
I20260810 18:45:08.430518 26102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:45:08.433224 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 25999
I20260810 18:45:08.433429 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387267874959-5011-0/minicluster-data/master-2/wal/instance
I20260810 18:45:08.434079 5011 external_mini_cluster.cc:1428] Setting key b9de4de7c64571aa273e84c7791d2100
I20260810 18:45:08.439054 26102 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42575 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:45:08 dist-test-slave-7kzw krb5kdc[25764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387506, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.252@KRBTEST.COM
Aug 10 18:45:08 dist-test-slave-7kzw krb5kdc[25764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387508, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.252@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:45:08.457937 26111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:08.442116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35792 (local address 127.4.228.254:42575)
0810 18:45:08.442757 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:08.442762 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:08.443161 (+ 399us) server_negotiation.cc:407] Connection header received
0810 18:45:08.443378 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:08.443383 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:08.443461 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:08.443646 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:08.444728 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:08.445459 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:08.446929 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:08.447362 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:08.447512 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:08.451174 (+ 3662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:08.451179 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:08.451182 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:08.451214 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:08.454556 (+ 3342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:08.455402 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:08.455408 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:08.455411 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:08.455571 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:08.456129 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:08.456133 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:08.456136 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:08.456474 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:08.456642 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:08.457305 (+ 663us) server_negotiation.cc:299] Negotiation successful
0810 18:45:08.457533 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":279,"threads_started":1}
I20260810 18:45:08.458998 26110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:08.436367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44548 (local address 127.4.228.252:33825)
0810 18:45:08.437251 (+ 884us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:08.437263 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:08.437307 (+ 44us) server_negotiation.cc:407] Connection header received
0810 18:45:08.438970 (+ 1663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:08.438995 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:08.439398 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:08.439728 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:08.441651 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:08.443648 (+ 1997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:08.445098 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:08.445767 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:08.445974 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:08.450712 (+ 4738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:08.450724 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:08.450741 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:08.450783 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:08.453951 (+ 3168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:08.455034 (+ 1083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:08.455045 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:08.455049 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:08.455248 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:08.455822 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:08.455829 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:08.455832 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:08.456555 (+ 723us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:08.456856 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:08.457271 (+ 415us) server_negotiation.cc:299] Negotiation successful
0810 18:45:08.457704 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":659,"thread_start_us":423,"threads_started":1}
I20260810 18:45:08.459017 26112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:08.442127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:42575 (local address 127.0.0.1:35792)
0810 18:45:08.443006 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:08.443052 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:08.443240 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:08.443664 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:08.443670 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:08.444294 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:45:08.444566 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:08.444576 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:08.445633 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:08.445643 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:08.446783 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:08.446797 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:08.447612 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:08.447620 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:08.447777 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:08.448410 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:45:08.448433 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:45:08.450980 (+ 2547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:45:08.454698 (+ 3718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:08.454707 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:08.454716 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:08.455237 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:08.455742 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:08.455746 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:08.455750 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:08.455979 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:08.456612 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:45:08.456617 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:45:08.456898 (+ 281us) client_negotiation.cc:769] Sending connection context
0810 18:45:08.457177 (+ 279us) client_negotiation.cc:240] Negotiation successful
0810 18:45:08.457949 (+ 772us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"mutex_wait_us":323,"thread_start_us":511,"threads_started":1}
I20260810 18:45:08.460661 26102 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.228.253" port: 43743 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:45:08 dist-test-slave-7kzw krb5kdc[25764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387508, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.252@KRBTEST.COM for kudu/127.4.228.253@KRBTEST.COM
Aug 10 18:45:08 dist-test-slave-7kzw krb5kdc[25764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387506, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:45:08.475029 25996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:08.461553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42358 (local address 127.4.228.253:43743)
0810 18:45:08.461754 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:08.461759 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:08.461987 (+ 228us) server_negotiation.cc:407] Connection header received
0810 18:45:08.462189 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:08.462197 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:08.462286 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:08.462364 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:08.463522 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:08.464306 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:08.465710 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:08.466130 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:08.466228 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:08.469524 (+ 3296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:08.469530 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:08.469534 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:08.469568 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:08.472473 (+ 2905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:08.473125 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:08.473131 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:08.473136 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:08.473275 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:08.473668 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:08.473673 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:08.473677 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:08.473934 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:08.474100 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:08.474561 (+ 461us) server_negotiation.cc:299] Negotiation successful
0810 18:45:08.474729 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 18:45:08.475101 26112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:08.461537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:43743 (local address 127.0.0.1:42358)
0810 18:45:08.461880 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:08.461899 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:08.462011 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:08.462653 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:08.462658 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:08.463145 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:45:08.463351 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:08.463365 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:08.464452 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:08.464455 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:08.465560 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:08.465576 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:08.466284 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:08.466293 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:08.466547 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:08.467173 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:45:08.467191 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:45:08.469316 (+ 2125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:45:08.472613 (+ 3297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:08.472619 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:08.472623 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:08.473002 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:08.473393 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:08.473398 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:08.473402 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:08.473548 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:08.474074 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:45:08.474081 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:45:08.474254 (+ 173us) client_negotiation.cc:769] Sending connection context
0810 18:45:08.474414 (+ 160us) client_negotiation.cc:240] Negotiation successful
0810 18:45:08.474737 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 18:45:08.476303 26102 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.228.252" port: 33825 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:45:08.477399 26111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:08.461969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35804 (local address 127.4.228.254:42575)
0810 18:45:08.462149 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:08.462153 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:08.462240 (+ 87us) server_negotiation.cc:407] Connection header received
0810 18:45:08.462401 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:08.462404 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:08.462512 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:08.462604 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:08.464162 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:08.464946 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:08.466625 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:08.467008 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:08.467165 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:08.471385 (+ 4220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:08.471398 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:08.471402 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:08.471435 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:08.474245 (+ 2810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:08.475096 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:08.475101 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:08.475104 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:08.475386 (+ 282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:08.475846 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:08.475858 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:08.475860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:08.476242 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:08.476548 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:08.476856 (+ 308us) server_negotiation.cc:299] Negotiation successful
0810 18:45:08.477092 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 18:45:08.484736 25872 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.252" port: 33825 } attempt: 4
Aug 10 18:45:08 dist-test-slave-7kzw krb5kdc[25764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387508, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.252@KRBTEST.COM for kudu/127.4.228.252@KRBTEST.COM
Aug 10 18:45:08 dist-test-slave-7kzw krb5kdc[25764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387506, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.253@KRBTEST.COM
Aug 10 18:45:08 dist-test-slave-7kzw krb5kdc[25764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387507, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for kudu/127.4.228.252@KRBTEST.COM
I20260810 18:45:08.493438 26112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:08.477028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:33825 (local address 127.0.0.1:44558)
0810 18:45:08.477323 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:08.477343 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:08.477564 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:08.477925 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:08.477930 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:08.478319 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:45:08.478566 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:08.478576 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:08.479849 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:08.479856 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:08.481313 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:08.481326 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:08.482348 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:08.482354 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:08.482640 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:08.483226 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:45:08.483252 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:45:08.486453 (+ 3201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:45:08.490651 (+ 4198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:08.490658 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:08.490664 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:08.491234 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:08.491686 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:08.491692 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:08.491696 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:08.491862 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:08.492427 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:45:08.492431 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:45:08.492570 (+ 139us) client_negotiation.cc:769] Sending connection context
0810 18:45:08.492726 (+ 156us) client_negotiation.cc:240] Negotiation successful
0810 18:45:08.493028 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177}
I20260810 18:45:08.493476 26110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:08.477178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44558 (local address 127.4.228.252:33825)
0810 18:45:08.477487 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:08.477494 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:08.477515 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:45:08.477694 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:08.477701 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:08.477761 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:08.477865 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:08.478694 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:08.479712 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:08.481462 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:08.482169 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:08.482334 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:08.486686 (+ 4352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:08.486694 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:08.486699 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:08.486736 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:08.490514 (+ 3778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:08.491375 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:08.491383 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:08.491387 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:08.491550 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:08.491993 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:08.492000 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:08.492002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:08.492305 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:08.492459 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:08.492803 (+ 344us) server_negotiation.cc:299] Negotiation successful
0810 18:45:08.493175 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":146,"server-negotiator.queue_time_us":209}
I20260810 18:45:08.498631 25996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:08.479207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42372 (local address 127.4.228.253:43743)
0810 18:45:08.479367 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:08.479372 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:08.479391 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:45:08.479542 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:08.479548 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:08.479613 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:08.479718 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:08.481276 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:08.482417 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:08.484319 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:08.484948 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:08.485125 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:08.492159 (+ 7034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:08.492166 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:08.492170 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:08.492214 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:08.495748 (+ 3534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:08.496410 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:08.496416 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:08.496420 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:08.496613 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:08.497075 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:08.497082 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:08.497085 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:08.497369 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:08.497669 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:08.497981 (+ 312us) server_negotiation.cc:299] Negotiation successful
0810 18:45:08.498225 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 18:45:08.501123 26102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8c3c67531434504afa24f3db2f87b4f: Bootstrap starting.
I20260810 18:45:08.501569 25877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:08.485307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:33825 (local address 127.0.0.1:44568)
0810 18:45:08.485494 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:08.485509 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:08.485647 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:08.486322 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:08.486325 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:08.486892 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:45:08.487093 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:08.487105 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:08.488361 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:08.488369 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:08.489466 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:08.489478 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:08.490217 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:08.490223 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:08.490493 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:08.491130 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:45:08.491159 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:45:08.494329 (+ 3170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:45:08.498621 (+ 4292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:08.498636 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:08.498640 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:08.499089 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:08.499543 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:08.499548 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:08.499552 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:08.499698 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:08.500559 (+ 861us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:45:08.500567 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:45:08.500745 (+ 178us) client_negotiation.cc:769] Sending connection context
0810 18:45:08.500904 (+ 159us) client_negotiation.cc:240] Negotiation successful
0810 18:45:08.501269 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260810 18:45:08.502830 26113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:08.485334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44568 (local address 127.4.228.252:33825)
0810 18:45:08.486005 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:08.486011 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:08.486034 (+ 23us) server_negotiation.cc:407] Connection header received
0810 18:45:08.486088 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:08.486093 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:08.486157 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:08.486277 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:08.487307 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:08.488197 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:08.489627 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:08.490069 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:08.490200 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:08.494637 (+ 4437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:08.494642 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:08.494645 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:08.494697 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:08.498311 (+ 3614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:08.499226 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:08.499231 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:08.499236 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:08.499396 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:08.499843 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:08.499848 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:08.499851 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:08.500421 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:08.500606 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:08.501011 (+ 405us) server_negotiation.cc:299] Negotiation successful
0810 18:45:08.502108 (+ 1097us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":319,"threads_started":1}
I20260810 18:45:08.503767 5011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:45:08.505188 26102 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8c3c67531434504afa24f3db2f87b4f: Neither blocks nor log segments found. Creating new log.
I20260810 18:45:08.506331 26102 log.cc:824] T 00000000000000000000000000000000 P d8c3c67531434504afa24f3db2f87b4f: Log is configured to *not* fsync() on all Append() calls
I20260810 18:45:08.509184 25872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38ff4aca9d0e4c37ae4ff101fb9d294b: Bootstrap starting.
I20260810 18:45:08.509366 26102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8c3c67531434504afa24f3db2f87b4f: No bootstrap required, opened a new log
I20260810 18:45:08.514191 25872 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38ff4aca9d0e4c37ae4ff101fb9d294b: Neither blocks nor log segments found. Creating new log.
I20260810 18:45:08.515679 25872 log.cc:824] T 00000000000000000000000000000000 P 38ff4aca9d0e4c37ae4ff101fb9d294b: Log is configured to *not* fsync() on all Append() calls
I20260810 18:45:08.517740 26102 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8c3c67531434504afa24f3db2f87b4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38ff4aca9d0e4c37ae4ff101fb9d294b" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42575 } } peers { permanent_uuid: "977723500379436084d4571a618805dd" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 43743 } } peers { permanent_uuid: "d8c3c67531434504afa24f3db2f87b4f" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 33825 } }
I20260810 18:45:08.518325 26102 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8c3c67531434504afa24f3db2f87b4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:45:08.518455 26102 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8c3c67531434504afa24f3db2f87b4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8c3c67531434504afa24f3db2f87b4f, State: Initialized, Role: FOLLOWER
I20260810 18:45:08.519309 26102 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8c3c67531434504afa24f3db2f87b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38ff4aca9d0e4c37ae4ff101fb9d294b" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42575 } } peers { permanent_uuid: "977723500379436084d4571a618805dd" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 43743 } } peers { permanent_uuid: "d8c3c67531434504afa24f3db2f87b4f" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 33825 } }
I20260810 18:45:08.520412 26125 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8c3c67531434504afa24f3db2f87b4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38ff4aca9d0e4c37ae4ff101fb9d294b" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42575 } } peers { permanent_uuid: "977723500379436084d4571a618805dd" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 43743 } } peers { permanent_uuid: "d8c3c67531434504afa24f3db2f87b4f" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 33825 } } }
I20260810 18:45:08.520952 26125 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8c3c67531434504afa24f3db2f87b4f [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:45:08.522090 26102 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8c3c67531434504afa24f3db2f87b4f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:45:08.526250 25872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38ff4aca9d0e4c37ae4ff101fb9d294b: No bootstrap required, opened a new log
I20260810 18:45:08.536149 26111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:08.517275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35812 (local address 127.4.228.254:42575)
0810 18:45:08.517428 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:08.517433 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:08.517893 (+ 460us) server_negotiation.cc:407] Connection header received
0810 18:45:08.518108 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:08.518113 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:08.518192 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:08.518305 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:08.520445 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:08.521469 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:08.523109 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:08.523674 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:08.523815 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:08.527726 (+ 3911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:08.527734 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:08.527738 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:08.527823 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:08.531686 (+ 3863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:08.532614 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:08.532622 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:08.532637 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:08.532809 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:08.533332 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:08.533342 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:08.533346 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:08.533755 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:08.534041 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:08.535494 (+ 1453us) server_negotiation.cc:299] Negotiation successful
0810 18:45:08.535751 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 18:45:08.540329 25872 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38ff4aca9d0e4c37ae4ff101fb9d294b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38ff4aca9d0e4c37ae4ff101fb9d294b" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42575 } } peers { permanent_uuid: "977723500379436084d4571a618805dd" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 43743 } } peers { permanent_uuid: "d8c3c67531434504afa24f3db2f87b4f" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 33825 } }
I20260810 18:45:08.540818 25872 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38ff4aca9d0e4c37ae4ff101fb9d294b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:45:08.541011 25872 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38ff4aca9d0e4c37ae4ff101fb9d294b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38ff4aca9d0e4c37ae4ff101fb9d294b, State: Initialized, Role: FOLLOWER
I20260810 18:45:08.541816 25872 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38ff4aca9d0e4c37ae4ff101fb9d294b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38ff4aca9d0e4c37ae4ff101fb9d294b" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42575 } } peers { permanent_uuid: "977723500379436084d4571a618805dd" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 43743 } } peers { permanent_uuid: "d8c3c67531434504afa24f3db2f87b4f" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 33825 } }
I20260810 18:45:08.550803 26133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38ff4aca9d0e4c37ae4ff101fb9d294b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38ff4aca9d0e4c37ae4ff101fb9d294b" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42575 } } peers { permanent_uuid: "977723500379436084d4571a618805dd" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 43743 } } peers { permanent_uuid: "d8c3c67531434504afa24f3db2f87b4f" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 33825 } } }
I20260810 18:45:08.551304 26133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38ff4aca9d0e4c37ae4ff101fb9d294b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:45:08.551970 25872 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38ff4aca9d0e4c37ae4ff101fb9d294b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:45:08.552191 25996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:08.517004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42384 (local address 127.4.228.253:43743)
0810 18:45:08.517197 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:08.517203 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:08.528804 (+ 11601us) server_negotiation.cc:407] Connection header received
0810 18:45:08.529937 (+ 1133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:08.529946 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:08.530048 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:08.530130 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:08.533253 (+ 3123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:08.534558 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:08.537433 (+ 2875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:08.538320 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:08.538444 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:08.543370 (+ 4926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:08.543379 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:08.543386 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:08.543433 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:08.547984 (+ 4551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:08.549099 (+ 1115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:08.549109 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:08.549115 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:08.549322 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:08.549851 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:08.549856 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:08.549859 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:08.550301 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:08.550631 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:08.551348 (+ 717us) server_negotiation.cc:299] Negotiation successful
0810 18:45:08.551665 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 18:45:08.554426 26113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:08.529462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44570 (local address 127.4.228.252:33825)
0810 18:45:08.529644 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:08.529652 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:08.533390 (+ 3738us) server_negotiation.cc:407] Connection header received
0810 18:45:08.534603 (+ 1213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:08.534612 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:08.534717 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:08.534789 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:08.537987 (+ 3198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:08.539107 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:08.541129 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:08.541799 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:08.541904 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:08.546239 (+ 4335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:08.546249 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:08.546258 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:08.546315 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:08.550732 (+ 4417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:08.551655 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:08.551660 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:08.551663 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:08.551816 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:08.552314 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:08.552321 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:08.552326 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:08.552706 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:08.552940 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:08.553600 (+ 660us) server_negotiation.cc:299] Negotiation successful
0810 18:45:08.553931 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260810 18:45:08.564018 26153 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d8c3c67531434504afa24f3db2f87b4f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:45:08.564713 26153 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:45:08.560393 25999 master_runner.cc:186] Error getting master registration for 127.4.228.254:42575: OK, instance_id { permanent_uuid: "38ff4aca9d0e4c37ae4ff101fb9d294b" instance_seqno: 1786387507339135 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 18:45:08.574862 25999 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260810 18:45:08.577893 25813 tablet.cc:2369] T 00000000000000000000000000000000 P 38ff4aca9d0e4c37ae4ff101fb9d294b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:45:08.582536 26164 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 38ff4aca9d0e4c37ae4ff101fb9d294b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:45:08.582808 26164 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:45:08.571790 25783 master_runner.cc:186] Error getting master registration for 127.4.228.253:43743: OK, instance_id { permanent_uuid: "977723500379436084d4571a618805dd" instance_seqno: 1786387507868372 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 18:45:08.583967 25783 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 18:45:08.601962 25989 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.252" port: 33825 } attempt: 5
Aug 10 18:45:08 dist-test-slave-7kzw krb5kdc[25764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387507, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.253@KRBTEST.COM for kudu/127.4.228.252@KRBTEST.COM
I20260810 18:45:08.625152 25993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:08.602587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:33825 (local address 127.0.0.1:44580)
0810 18:45:08.602874 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:08.602897 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:08.603020 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:08.603419 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:08.603425 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:08.603986 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:45:08.604213 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:08.604226 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:08.605828 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:08.605834 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:08.607367 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:08.607384 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:08.608516 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:08.608523 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:08.608735 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:08.614189 (+ 5454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:45:08.614216 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:45:08.617496 (+ 3280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:45:08.621870 (+ 4374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:08.621876 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:08.621880 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:08.622578 (+ 698us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:08.623057 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:08.623062 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:08.623074 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:08.623254 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:08.624020 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:45:08.624027 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:45:08.624220 (+ 193us) client_negotiation.cc:769] Sending connection context
0810 18:45:08.624384 (+ 164us) client_negotiation.cc:240] Negotiation successful
0810 18:45:08.624702 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155}
I20260810 18:45:08.626658 26113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:08.602700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44580 (local address 127.4.228.252:33825)
0810 18:45:08.602886 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:08.602893 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:08.602963 (+ 70us) server_negotiation.cc:407] Connection header received
0810 18:45:08.603161 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:08.603166 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:08.603255 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:08.603365 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:08.604403 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:08.605666 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:08.607573 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:08.608285 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:08.608430 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:08.617770 (+ 9340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:08.617778 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:08.617784 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:08.617836 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:08.621697 (+ 3861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:08.622729 (+ 1032us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:08.622735 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:08.622739 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:08.622920 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:08.623473 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:08.623481 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:08.623490 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:08.623877 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:08.624064 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:08.625949 (+ 1885us) server_negotiation.cc:299] Negotiation successful
0810 18:45:08.626225 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 18:45:08.632972 25989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 977723500379436084d4571a618805dd: Bootstrap starting.
I20260810 18:45:08.637344 25989 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 977723500379436084d4571a618805dd: Neither blocks nor log segments found. Creating new log.
I20260810 18:45:08.638753 25989 log.cc:824] T 00000000000000000000000000000000 P 977723500379436084d4571a618805dd: Log is configured to *not* fsync() on all Append() calls
I20260810 18:45:08.642195 25989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 977723500379436084d4571a618805dd: No bootstrap required, opened a new log
I20260810 18:45:08.653628 25989 raft_consensus.cc:348] T 00000000000000000000000000000000 P 977723500379436084d4571a618805dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38ff4aca9d0e4c37ae4ff101fb9d294b" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42575 } } peers { permanent_uuid: "977723500379436084d4571a618805dd" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 43743 } } peers { permanent_uuid: "d8c3c67531434504afa24f3db2f87b4f" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 33825 } }
I20260810 18:45:08.654089 25989 raft_consensus.cc:374] T 00000000000000000000000000000000 P 977723500379436084d4571a618805dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:45:08.654219 25989 raft_consensus.cc:729] T 00000000000000000000000000000000 P 977723500379436084d4571a618805dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 977723500379436084d4571a618805dd, State: Initialized, Role: FOLLOWER
I20260810 18:45:08.654944 25989 consensus_queue.cc:260] T 00000000000000000000000000000000 P 977723500379436084d4571a618805dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38ff4aca9d0e4c37ae4ff101fb9d294b" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42575 } } peers { permanent_uuid: "977723500379436084d4571a618805dd" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 43743 } } peers { permanent_uuid: "d8c3c67531434504afa24f3db2f87b4f" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 33825 } }
I20260810 18:45:08.655992 26169 sys_catalog.cc:455] T 00000000000000000000000000000000 P 977723500379436084d4571a618805dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38ff4aca9d0e4c37ae4ff101fb9d294b" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42575 } } peers { permanent_uuid: "977723500379436084d4571a618805dd" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 43743 } } peers { permanent_uuid: "d8c3c67531434504afa24f3db2f87b4f" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 33825 } } }
I20260810 18:45:08.656689 26169 sys_catalog.cc:458] T 00000000000000000000000000000000 P 977723500379436084d4571a618805dd [sys.catalog]: This master's current role is: FOLLOWER
W20260810 18:45:08.657406 26037 tablet.cc:2369] T 00000000000000000000000000000000 P d8c3c67531434504afa24f3db2f87b4f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:45:08.657457 25989 sys_catalog.cc:564] T 00000000000000000000000000000000 P 977723500379436084d4571a618805dd [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:45:08.687022 25883 master_runner.cc:263] No leader master found from master 977723500379436084d4571a618805dd
I20260810 18:45:08.687209 25883 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260810 18:45:08.690054 26181 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 977723500379436084d4571a618805dd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:45:08.690307 26181 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:45:08.713049 26169 raft_consensus.cc:482] T 00000000000000000000000000000000 P 977723500379436084d4571a618805dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:45:08.713390 26169 raft_consensus.cc:504] T 00000000000000000000000000000000 P 977723500379436084d4571a618805dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38ff4aca9d0e4c37ae4ff101fb9d294b" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42575 } } peers { permanent_uuid: "977723500379436084d4571a618805dd" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 43743 } } peers { permanent_uuid: "d8c3c67531434504afa24f3db2f87b4f" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 33825 } }
I20260810 18:45:08.717593 26169 leader_election.cc:288] T 00000000000000000000000000000000 P 977723500379436084d4571a618805dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38ff4aca9d0e4c37ae4ff101fb9d294b (127.4.228.254:42575), d8c3c67531434504afa24f3db2f87b4f (127.4.228.252:33825)
I20260810 18:45:08.719125 25843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "977723500379436084d4571a618805dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38ff4aca9d0e4c37ae4ff101fb9d294b" is_pre_election: true
I20260810 18:45:08.719853 25843 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 38ff4aca9d0e4c37ae4ff101fb9d294b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 977723500379436084d4571a618805dd in term 0.
I20260810 18:45:08.720508 26068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "977723500379436084d4571a618805dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8c3c67531434504afa24f3db2f87b4f" is_pre_election: true
I20260810 18:45:08.721120 26068 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d8c3c67531434504afa24f3db2f87b4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 977723500379436084d4571a618805dd in term 0.
I20260810 18:45:08.722000 25906 leader_election.cc:302] T 00000000000000000000000000000000 P 977723500379436084d4571a618805dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38ff4aca9d0e4c37ae4ff101fb9d294b, 977723500379436084d4571a618805dd; no voters:
I20260810 18:45:08.722903 26169 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 977723500379436084d4571a618805dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:45:08.723053 26169 raft_consensus.cc:482] T 00000000000000000000000000000000 P 977723500379436084d4571a618805dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:45:08.723155 26169 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 977723500379436084d4571a618805dd [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:45:08.726859 26169 raft_consensus.cc:504] T 00000000000000000000000000000000 P 977723500379436084d4571a618805dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38ff4aca9d0e4c37ae4ff101fb9d294b" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42575 } } peers { permanent_uuid: "977723500379436084d4571a618805dd" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 43743 } } peers { permanent_uuid: "d8c3c67531434504afa24f3db2f87b4f" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 33825 } }
I20260810 18:45:08.728073 25843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "977723500379436084d4571a618805dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38ff4aca9d0e4c37ae4ff101fb9d294b"
I20260810 18:45:08.728308 25843 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38ff4aca9d0e4c37ae4ff101fb9d294b [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:45:08.731822 25843 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 38ff4aca9d0e4c37ae4ff101fb9d294b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 977723500379436084d4571a618805dd in term 1.
I20260810 18:45:08.732353 25906 leader_election.cc:302] T 00000000000000000000000000000000 P 977723500379436084d4571a618805dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38ff4aca9d0e4c37ae4ff101fb9d294b, 977723500379436084d4571a618805dd; no voters:
I20260810 18:45:08.734809 26169 leader_election.cc:288] T 00000000000000000000000000000000 P 977723500379436084d4571a618805dd [CANDIDATE]: Term 1 election: Requested vote from peers 38ff4aca9d0e4c37ae4ff101fb9d294b (127.4.228.254:42575), d8c3c67531434504afa24f3db2f87b4f (127.4.228.252:33825)
I20260810 18:45:08.738397 26169 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 977723500379436084d4571a618805dd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:45:08.738909 26169 raft_consensus.cc:686] T 00000000000000000000000000000000 P 977723500379436084d4571a618805dd [term 1 LEADER]: Becoming Leader. State: Replica: 977723500379436084d4571a618805dd, State: Running, Role: LEADER
I20260810 18:45:08.739817 26169 consensus_queue.cc:237] T 00000000000000000000000000000000 P 977723500379436084d4571a618805dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38ff4aca9d0e4c37ae4ff101fb9d294b" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42575 } } peers { permanent_uuid: "977723500379436084d4571a618805dd" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 43743 } } peers { permanent_uuid: "d8c3c67531434504afa24f3db2f87b4f" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 33825 } }
I20260810 18:45:08.750077 26068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "977723500379436084d4571a618805dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8c3c67531434504afa24f3db2f87b4f"
I20260810 18:45:08.750337 26068 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8c3c67531434504afa24f3db2f87b4f [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:45:08.754287 26068 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d8c3c67531434504afa24f3db2f87b4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 977723500379436084d4571a618805dd in term 1.
I20260810 18:45:08.758196 26183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 977723500379436084d4571a618805dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 977723500379436084d4571a618805dd. Latest consensus state: current_term: 1 leader_uuid: "977723500379436084d4571a618805dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38ff4aca9d0e4c37ae4ff101fb9d294b" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42575 } } peers { permanent_uuid: "977723500379436084d4571a618805dd" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 43743 } } peers { permanent_uuid: "d8c3c67531434504afa24f3db2f87b4f" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 33825 } } }
I20260810 18:45:08.758814 26183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 977723500379436084d4571a618805dd [sys.catalog]: This master's current role is: LEADER
I20260810 18:45:08.765800 26186 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:45:08.768359 26186 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:45:08.789005 26068 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d8c3c67531434504afa24f3db2f87b4f [term 1 FOLLOWER]: Refusing update from remote peer 977723500379436084d4571a618805dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:45:08.790233 26183 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 977723500379436084d4571a618805dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8c3c67531434504afa24f3db2f87b4f" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 33825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:45:08.798142 25843 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 38ff4aca9d0e4c37ae4ff101fb9d294b [term 1 FOLLOWER]: Refusing update from remote peer 977723500379436084d4571a618805dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:45:08.802927 26183 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 977723500379436084d4571a618805dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "38ff4aca9d0e4c37ae4ff101fb9d294b" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:45:08.823613 26125 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8c3c67531434504afa24f3db2f87b4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 977723500379436084d4571a618805dd. Latest consensus state: current_term: 1 leader_uuid: "977723500379436084d4571a618805dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38ff4aca9d0e4c37ae4ff101fb9d294b" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42575 } } peers { permanent_uuid: "977723500379436084d4571a618805dd" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 43743 } } peers { permanent_uuid: "d8c3c67531434504afa24f3db2f87b4f" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 33825 } } }
I20260810 18:45:08.823938 26125 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8c3c67531434504afa24f3db2f87b4f [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:45:08.828112 26187 mvcc.cc:205] Tried to move back new op lower bound from 7317043235988144128 to 7317043235810856960. Current Snapshot: MvccSnapshot[applied={T|T < 7317043235988144128}]
I20260810 18:45:08.834597 26188 mvcc.cc:205] Tried to move back new op lower bound from 7317043235988144128 to 7317043235810856960. Current Snapshot: MvccSnapshot[applied={T|T < 7317043235988144128}]
I20260810 18:45:08.835978 26133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38ff4aca9d0e4c37ae4ff101fb9d294b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 977723500379436084d4571a618805dd. Latest consensus state: current_term: 1 leader_uuid: "977723500379436084d4571a618805dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38ff4aca9d0e4c37ae4ff101fb9d294b" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42575 } } peers { permanent_uuid: "977723500379436084d4571a618805dd" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 43743 } } peers { permanent_uuid: "d8c3c67531434504afa24f3db2f87b4f" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 33825 } } }
I20260810 18:45:08.836316 26133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38ff4aca9d0e4c37ae4ff101fb9d294b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:45:08.846853 26125 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8c3c67531434504afa24f3db2f87b4f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "977723500379436084d4571a618805dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38ff4aca9d0e4c37ae4ff101fb9d294b" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42575 } } peers { permanent_uuid: "977723500379436084d4571a618805dd" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 43743 } } peers { permanent_uuid: "d8c3c67531434504afa24f3db2f87b4f" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 33825 } } }
I20260810 18:45:08.847164 26125 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8c3c67531434504afa24f3db2f87b4f [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:45:08.847028 26133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38ff4aca9d0e4c37ae4ff101fb9d294b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "977723500379436084d4571a618805dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38ff4aca9d0e4c37ae4ff101fb9d294b" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42575 } } peers { permanent_uuid: "977723500379436084d4571a618805dd" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 43743 } } peers { permanent_uuid: "d8c3c67531434504afa24f3db2f87b4f" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 33825 } } }
I20260810 18:45:08.847295 26133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38ff4aca9d0e4c37ae4ff101fb9d294b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:45:08.853346 26186 catalog_manager.cc:1347] Generated new cluster ID: 636b1184e2ef4f0286535d68209d14c6
I20260810 18:45:08.853467 26186 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:45:08.877696 26204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 977723500379436084d4571a618805dd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "977723500379436084d4571a618805dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38ff4aca9d0e4c37ae4ff101fb9d294b" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42575 } } peers { permanent_uuid: "977723500379436084d4571a618805dd" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 43743 } } peers { permanent_uuid: "d8c3c67531434504afa24f3db2f87b4f" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 33825 } } }
I20260810 18:45:08.886922 26204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 977723500379436084d4571a618805dd [sys.catalog]: This master's current role is: LEADER
I20260810 18:45:08.887027 26169 sys_catalog.cc:455] T 00000000000000000000000000000000 P 977723500379436084d4571a618805dd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "977723500379436084d4571a618805dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38ff4aca9d0e4c37ae4ff101fb9d294b" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42575 } } peers { permanent_uuid: "977723500379436084d4571a618805dd" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 43743 } } peers { permanent_uuid: "d8c3c67531434504afa24f3db2f87b4f" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 33825 } } }
I20260810 18:45:08.888607 26169 sys_catalog.cc:458] T 00000000000000000000000000000000 P 977723500379436084d4571a618805dd [sys.catalog]: This master's current role is: LEADER
I20260810 18:45:08.898186 26186 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:45:08.900601 26186 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:45:08.919941 26186 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 977723500379436084d4571a618805dd: Generated new TSK 0
I20260810 18:45:08.921231 26186 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:45:09.566949 26153 catalog_manager.cc:1259] Loaded cluster ID: 636b1184e2ef4f0286535d68209d14c6
I20260810 18:45:09.567094 26153 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d8c3c67531434504afa24f3db2f87b4f: loading cluster ID for follower catalog manager: success
I20260810 18:45:09.569620 26153 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d8c3c67531434504afa24f3db2f87b4f: acquiring CA information for follower catalog manager: success
I20260810 18:45:09.571514 26153 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d8c3c67531434504afa24f3db2f87b4f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 18:45:09.585434 26164 catalog_manager.cc:1259] Loaded cluster ID: 636b1184e2ef4f0286535d68209d14c6
I20260810 18:45:09.585562 26164 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 38ff4aca9d0e4c37ae4ff101fb9d294b: loading cluster ID for follower catalog manager: success
I20260810 18:45:09.588634 26164 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 38ff4aca9d0e4c37ae4ff101fb9d294b: acquiring CA information for follower catalog manager: success
I20260810 18:45:09.590530 26164 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 38ff4aca9d0e4c37ae4ff101fb9d294b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 18:45:10.104877 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 25783
I20260810 18:45:10.106688 25866 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:45:10.221077 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 25783
I20260810 18:45:10.259749 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 25883
I20260810 18:45:10.261086 25982 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:45:10.370409 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 25883
I20260810 18:45:10.402113 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 25999
I20260810 18:45:10.403403 26095 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:45:10.512171 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 25999
2026-08-10T18:45:10Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5267 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:45:10 dist-test-slave-7kzw krb5kdc[26221](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:45:10 dist-test-slave-7kzw krb5kdc[26221](info): set up 2 sockets
Aug 10 18:45:10 dist-test-slave-7kzw krb5kdc[26221](info): commencing operation
krb5kdc: starting...
W20260810 18:45:11.851969 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.287s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:45:12 dist-test-slave-7kzw krb5kdc[26221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387512, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:45:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:45:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.228.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
I20260810 18:45:12.156616 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:34589
--webserver_interface=127.4.228.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:37683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:34589
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:45:12.507910 26246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:45:12.508220 26246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:45:12.508388 26246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:45:12.515744 26246 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:45:12.515851 26246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:45:12.515939 26246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:45:12.516026 26246 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:45:12.516121 26246 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:45:12.526079 26246 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:37683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:34589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:34589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:45:12.527583 26246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:45:12.529035 26246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:45:12.541571 26254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:12.541827 26258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:12.542063 26255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:45:12.542744 26246 server_base.cc:1034] running on GCE node
I20260810 18:45:12.543625 26246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:45:12.545284 26246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:45:12.546612 26246 hybrid_clock.cc:648] HybridClock initialized: now 1786387512546541 us; error 39 us; skew 500 ppm
I20260810 18:45:12.547024 26246 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:45:12 dist-test-slave-7kzw krb5kdc[26221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387512, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:45:12.550518 26246 init.cc:377] Logged in from keytab as kudu/127.4.228.254@KRBTEST.COM (short username kudu)
I20260810 18:45:12.552163 26246 webserver.cc:460] Webserver started at http://127.4.228.254:43739/ using document root <none> and password file <none>
I20260810 18:45:12.552942 26246 fs_manager.cc:359] Metadata directory not provided
I20260810 18:45:12.553119 26246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:45:12.553496 26246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:45:12.556969 26246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "8cffe712e6bf4665813e8d61658b28a3"
format_stamp: "Formatted at 2026-08-10 18:45:12 on dist-test-slave-7kzw"
server_key: "7dbdb2530eaab83968e03273f9e5563b"
server_key_iv: "88322ec55d603510f5439f7b84be4b13"
server_key_version: "encryptionkey@0"
I20260810 18:45:12.557835 26246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "8cffe712e6bf4665813e8d61658b28a3"
format_stamp: "Formatted at 2026-08-10 18:45:12 on dist-test-slave-7kzw"
server_key: "7dbdb2530eaab83968e03273f9e5563b"
server_key_iv: "88322ec55d603510f5439f7b84be4b13"
server_key_version: "encryptionkey@0"
I20260810 18:45:12.563586 26246 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 18:45:12.567874 26267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:12.569459 26246 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260810 18:45:12.569716 26246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "8cffe712e6bf4665813e8d61658b28a3"
format_stamp: "Formatted at 2026-08-10 18:45:12 on dist-test-slave-7kzw"
server_key: "7dbdb2530eaab83968e03273f9e5563b"
server_key_iv: "88322ec55d603510f5439f7b84be4b13"
server_key_version: "encryptionkey@0"
I20260810 18:45:12.569957 26246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:45:12.592160 26246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:45:12.596762 26246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:45:12.597093 26246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:45:12.617455 26246 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:34589
I20260810 18:45:12.617479 26339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:34589 every 8 connection(s)
I20260810 18:45:12.619016 26246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:45:12.623024 26340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:45:12.626946 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 26246
I20260810 18:45:12.627207 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:45:12.627942 5011 external_mini_cluster.cc:1428] Setting key 579798792480921342ca1859d3cf7c11
I20260810 18:45:12.634684 26340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cffe712e6bf4665813e8d61658b28a3: Bootstrap starting.
I20260810 18:45:12.638494 26340 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8cffe712e6bf4665813e8d61658b28a3: Neither blocks nor log segments found. Creating new log.
I20260810 18:45:12.639763 26340 log.cc:824] T 00000000000000000000000000000000 P 8cffe712e6bf4665813e8d61658b28a3: Log is configured to *not* fsync() on all Append() calls
I20260810 18:45:12.642835 26340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cffe712e6bf4665813e8d61658b28a3: No bootstrap required, opened a new log
Aug 10 18:45:12 dist-test-slave-7kzw krb5kdc[26221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387512, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:45:12.653908 26340 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8cffe712e6bf4665813e8d61658b28a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cffe712e6bf4665813e8d61658b28a3" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 34589 } }
I20260810 18:45:12.654373 26340 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8cffe712e6bf4665813e8d61658b28a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:45:12.654408 26345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:12.630067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59984 (local address 127.4.228.254:34589)
0810 18:45:12.631017 (+ 950us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:12.631030 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:12.631083 (+ 53us) server_negotiation.cc:407] Connection header received
0810 18:45:12.633118 (+ 2035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:12.633144 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:12.633556 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:12.633899 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:12.635980 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:12.637365 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:12.639399 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:12.640201 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:12.640465 (+ 264us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:12.645337 (+ 4872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:12.645347 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:12.645367 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:12.645431 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:12.649648 (+ 4217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:12.650398 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:12.650412 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:12.650421 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:12.650669 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:12.651053 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:12.651063 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:12.651067 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:12.651844 (+ 777us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:12.652197 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:12.652575 (+ 378us) server_negotiation.cc:299] Negotiation successful
0810 18:45:12.653032 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":439,"threads_started":1}
I20260810 18:45:12.654546 26340 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8cffe712e6bf4665813e8d61658b28a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cffe712e6bf4665813e8d61658b28a3, State: Initialized, Role: FOLLOWER
I20260810 18:45:12.655253 26340 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cffe712e6bf4665813e8d61658b28a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cffe712e6bf4665813e8d61658b28a3" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 34589 } }
I20260810 18:45:12.655544 26340 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8cffe712e6bf4665813e8d61658b28a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:45:12.655759 26340 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cffe712e6bf4665813e8d61658b28a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:45:12.655983 26340 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8cffe712e6bf4665813e8d61658b28a3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:45:12.658785 5011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:45:12.658751 26340 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cffe712e6bf4665813e8d61658b28a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cffe712e6bf4665813e8d61658b28a3" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 34589 } }
I20260810 18:45:12.659301 26340 leader_election.cc:302] T 00000000000000000000000000000000 P 8cffe712e6bf4665813e8d61658b28a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8cffe712e6bf4665813e8d61658b28a3; no voters:
I20260810 18:45:12.659971 26340 leader_election.cc:288] T 00000000000000000000000000000000 P 8cffe712e6bf4665813e8d61658b28a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:45:12.660080 26347 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cffe712e6bf4665813e8d61658b28a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:45:12.661425 26347 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8cffe712e6bf4665813e8d61658b28a3 [term 1 LEADER]: Becoming Leader. State: Replica: 8cffe712e6bf4665813e8d61658b28a3, State: Running, Role: LEADER
I20260810 18:45:12.662041 26347 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cffe712e6bf4665813e8d61658b28a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cffe712e6bf4665813e8d61658b28a3" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 34589 } }
I20260810 18:45:12.662485 26340 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8cffe712e6bf4665813e8d61658b28a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:45:12.665760 26350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cffe712e6bf4665813e8d61658b28a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cffe712e6bf4665813e8d61658b28a3. Latest consensus state: current_term: 1 leader_uuid: "8cffe712e6bf4665813e8d61658b28a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cffe712e6bf4665813e8d61658b28a3" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 34589 } } }
I20260810 18:45:12.666168 26350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cffe712e6bf4665813e8d61658b28a3 [sys.catalog]: This master's current role is: LEADER
I20260810 18:45:12.665885 26349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cffe712e6bf4665813e8d61658b28a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8cffe712e6bf4665813e8d61658b28a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cffe712e6bf4665813e8d61658b28a3" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 34589 } } }
I20260810 18:45:12.666307 26349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cffe712e6bf4665813e8d61658b28a3 [sys.catalog]: This master's current role is: LEADER
I20260810 18:45:12.667963 26356 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:45:12.674280 26356 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:45:12.683140 26356 catalog_manager.cc:1347] Generated new cluster ID: b1139153ba1a46caa6ad04f15da13b35
I20260810 18:45:12.683320 26356 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:45:12.691545 26356 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:45:12.693357 26356 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:45:12.706660 26356 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8cffe712e6bf4665813e8d61658b28a3: Generated new TSK 0
I20260810 18:45:12.707724 26356 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:45:12.721025 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 26246
I20260810 18:45:12.722359 26333 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:45:12.813957 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 26246
2026-08-10T18:45:12Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2304 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:45:12 dist-test-slave-7kzw krb5kdc[26383](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:45:12 dist-test-slave-7kzw krb5kdc[26383](info): set up 2 sockets
Aug 10 18:45:12 dist-test-slave-7kzw krb5kdc[26383](info): commencing operation
krb5kdc: starting...
W20260810 18:45:14.160290 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.289s user 0.002s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:45:14 dist-test-slave-7kzw krb5kdc[26383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:45:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:45:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.228.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
I20260810 18:45:14.462252 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:45083
--webserver_interface=127.4.228.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:40315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:45083
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:45:14.826260 26404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:45:14.826694 26404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:45:14.826885 26404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:45:14.834282 26404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:45:14.834396 26404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:45:14.834566 26404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:45:14.834687 26404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:45:14.834779 26404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:45:14.844911 26404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:40315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:45083
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:45083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:45:14.846323 26404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:45:14.847749 26404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:45:14.860108 26411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:14.860080 26413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:14.860267 26418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:45:14.860917 26404 server_base.cc:1034] running on GCE node
I20260810 18:45:14.861712 26404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:45:14.863281 26404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:45:14.864506 26404 hybrid_clock.cc:648] HybridClock initialized: now 1786387514864425 us; error 75 us; skew 500 ppm
I20260810 18:45:14.864898 26404 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:45:14 dist-test-slave-7kzw krb5kdc[26383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387514, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:45:14.871719 26404 init.cc:377] Logged in from keytab as kudu/127.4.228.254@KRBTEST.COM (short username kudu)
I20260810 18:45:14.873376 26404 webserver.cc:460] Webserver started at http://127.4.228.254:33247/ using document root <none> and password file <none>
I20260810 18:45:14.874094 26404 fs_manager.cc:359] Metadata directory not provided
I20260810 18:45:14.874213 26404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:45:14.874542 26404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:45:14.877761 26404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "346a38b450e0436191ddbba9e03b1198"
format_stamp: "Formatted at 2026-08-10 18:45:14 on dist-test-slave-7kzw"
server_key: "0776f4b220826e5549b8e15679d14529"
server_key_iv: "f27d06aa6bfe6f501895bb753c210983"
server_key_version: "encryptionkey@0"
I20260810 18:45:14.878614 26404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "346a38b450e0436191ddbba9e03b1198"
format_stamp: "Formatted at 2026-08-10 18:45:14 on dist-test-slave-7kzw"
server_key: "0776f4b220826e5549b8e15679d14529"
server_key_iv: "f27d06aa6bfe6f501895bb753c210983"
server_key_version: "encryptionkey@0"
I20260810 18:45:14.884192 26404 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260810 18:45:14.887969 26427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:14.889423 26404 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:45:14.889647 26404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "346a38b450e0436191ddbba9e03b1198"
format_stamp: "Formatted at 2026-08-10 18:45:14 on dist-test-slave-7kzw"
server_key: "0776f4b220826e5549b8e15679d14529"
server_key_iv: "f27d06aa6bfe6f501895bb753c210983"
server_key_version: "encryptionkey@0"
I20260810 18:45:14.889866 26404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:45:14.926440 26404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:45:14.931258 26404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:45:14.931610 26404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:45:14.951414 26404 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:45083
I20260810 18:45:14.951447 26498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:45083 every 8 connection(s)
I20260810 18:45:14.953038 26404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:45:14.957129 26499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:45:14.962031 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 26404
I20260810 18:45:14.962288 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:45:14.963049 5011 external_mini_cluster.cc:1428] Setting key 2d5cde980aa8447f6392cb7c53fb6f03
I20260810 18:45:14.969331 26499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198: Bootstrap starting.
I20260810 18:45:14.973161 26499 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198: Neither blocks nor log segments found. Creating new log.
I20260810 18:45:14.974494 26499 log.cc:824] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198: Log is configured to *not* fsync() on all Append() calls
I20260810 18:45:14.978602 26499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198: No bootstrap required, opened a new log
Aug 10 18:45:14 dist-test-slave-7kzw krb5kdc[26383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:45:14.987056 26499 raft_consensus.cc:348] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "346a38b450e0436191ddbba9e03b1198" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45083 } }
I20260810 18:45:14.987488 26499 raft_consensus.cc:374] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:45:14.987622 26499 raft_consensus.cc:729] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 346a38b450e0436191ddbba9e03b1198, State: Initialized, Role: FOLLOWER
I20260810 18:45:14.988260 26499 consensus_queue.cc:260] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "346a38b450e0436191ddbba9e03b1198" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45083 } }
I20260810 18:45:14.988521 26499 raft_consensus.cc:388] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:45:14.988688 26499 raft_consensus.cc:482] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:45:14.988904 26499 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:45:14.990921 26503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:14.965238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50570 (local address 127.4.228.254:45083)
0810 18:45:14.966171 (+ 933us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:14.966186 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:14.966230 (+ 44us) server_negotiation.cc:407] Connection header received
0810 18:45:14.968461 (+ 2231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:14.968487 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:14.968894 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:14.969228 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:14.971204 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:14.972833 (+ 1629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:14.974833 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:14.976310 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:14.976478 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:14.981591 (+ 5113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:14.981600 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:14.981617 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:14.981664 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:14.985366 (+ 3702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:14.986362 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:14.986370 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:14.986378 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:14.986665 (+ 287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:14.987163 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:14.987177 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:14.987180 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:14.988067 (+ 887us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:14.988456 (+ 389us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:14.989067 (+ 611us) server_negotiation.cc:299] Negotiation successful
0810 18:45:14.989593 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":694,"thread_start_us":438,"threads_started":1}
I20260810 18:45:14.991775 26499 raft_consensus.cc:504] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "346a38b450e0436191ddbba9e03b1198" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45083 } }
I20260810 18:45:14.992386 26499 leader_election.cc:302] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 346a38b450e0436191ddbba9e03b1198; no voters:
I20260810 18:45:14.993185 26499 leader_election.cc:288] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:45:14.993309 26505 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:45:14.994151 26505 raft_consensus.cc:686] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [term 1 LEADER]: Becoming Leader. State: Replica: 346a38b450e0436191ddbba9e03b1198, State: Running, Role: LEADER
I20260810 18:45:14.995319 26505 consensus_queue.cc:237] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "346a38b450e0436191ddbba9e03b1198" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45083 } }
I20260810 18:45:14.996657 26499 sys_catalog.cc:564] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:45:14.998663 26507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "346a38b450e0436191ddbba9e03b1198" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "346a38b450e0436191ddbba9e03b1198" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45083 } } }
I20260810 18:45:14.999069 26507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [sys.catalog]: This master's current role is: LEADER
I20260810 18:45:15.001856 26512 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:45:15.001617 26506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 346a38b450e0436191ddbba9e03b1198. Latest consensus state: current_term: 1 leader_uuid: "346a38b450e0436191ddbba9e03b1198" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "346a38b450e0436191ddbba9e03b1198" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45083 } } }
I20260810 18:45:15.002028 26506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [sys.catalog]: This master's current role is: LEADER
I20260810 18:45:15.008554 26512 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:45:15.017746 26512 catalog_manager.cc:1347] Generated new cluster ID: 17650ec17c2247408e130a2363f61f0b
I20260810 18:45:15.017870 26512 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:45:15.031495 26512 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:45:15.033613 26512 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:45:15.043819 26512 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198: Generated new TSK 0
I20260810 18:45:15.044818 26512 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.228.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab.
Entry for principal kudu/127.4.228.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab.
I20260810 18:45:15.127552 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.193:0
--local_ip_for_outbound_sockets=127.4.228.193
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:45083
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:40315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:45:15.486529 26541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:45:15.486922 26541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:45:15.487107 26541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:45:15.494854 26541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:45:15.495011 26541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:45:15.495190 26541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.193
I20260810 18:45:15.505803 26541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:40315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:45083
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.193
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:45:15.507329 26541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:45:15.508736 26541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:45:15.522634 26549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:15.522590 26552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:15.522656 26550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:45:15.523607 26541 server_base.cc:1034] running on GCE node
I20260810 18:45:15.524226 26541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:45:15.525206 26541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:45:15.526434 26541 hybrid_clock.cc:648] HybridClock initialized: now 1786387515526372 us; error 46 us; skew 500 ppm
I20260810 18:45:15.526850 26541 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:45:15 dist-test-slave-7kzw krb5kdc[26383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387515, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:45:15.530231 26541 init.cc:377] Logged in from keytab as kudu/127.4.228.193@KRBTEST.COM (short username kudu)
I20260810 18:45:15.531872 26541 webserver.cc:460] Webserver started at http://127.4.228.193:41535/ using document root <none> and password file <none>
I20260810 18:45:15.532578 26541 fs_manager.cc:359] Metadata directory not provided
I20260810 18:45:15.532684 26541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:45:15.532928 26541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:45:15.535944 26541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/data/instance:
uuid: "6114ee077fd140739ff7a63019042a8f"
format_stamp: "Formatted at 2026-08-10 18:45:15 on dist-test-slave-7kzw"
server_key: "030db39962bc74b50adf6bed4baf5176"
server_key_iv: "e4870105dfe0384426f9f00d9ed082d9"
server_key_version: "encryptionkey@0"
I20260810 18:45:15.536685 26541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance:
uuid: "6114ee077fd140739ff7a63019042a8f"
format_stamp: "Formatted at 2026-08-10 18:45:15 on dist-test-slave-7kzw"
server_key: "030db39962bc74b50adf6bed4baf5176"
server_key_iv: "e4870105dfe0384426f9f00d9ed082d9"
server_key_version: "encryptionkey@0"
I20260810 18:45:15.542202 26541 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 18:45:15.546159 26563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:15.547636 26541 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:45:15.547827 26541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/wal
uuid: "6114ee077fd140739ff7a63019042a8f"
format_stamp: "Formatted at 2026-08-10 18:45:15 on dist-test-slave-7kzw"
server_key: "030db39962bc74b50adf6bed4baf5176"
server_key_iv: "e4870105dfe0384426f9f00d9ed082d9"
server_key_version: "encryptionkey@0"
I20260810 18:45:15.548033 26541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:45:15.585073 26541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:45:15.591470 26541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:45:15.591732 26541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:45:15.592741 26541 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:45:15.594238 26541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:45:15.594317 26541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:15.594412 26541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:45:15.594457 26541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:15.630599 26541 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.193:42335
I20260810 18:45:15.630726 26722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.193:42335 every 8 connection(s)
I20260810 18:45:15.632001 26541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
I20260810 18:45:15.640381 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 26541
I20260810 18:45:15.640619 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance
I20260810 18:45:15.641337 5011 external_mini_cluster.cc:1428] Setting key 292799b348965e9f20f541c761857b5c
Aug 10 18:45:15 dist-test-slave-7kzw krb5kdc[26383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387515, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:45:15.653556 26727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:15.634801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:55605 (local address 127.4.228.254:45083)
0810 18:45:15.635281 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:15.635286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:15.636108 (+ 822us) server_negotiation.cc:407] Connection header received
0810 18:45:15.638000 (+ 1892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:15.638006 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:15.638110 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:15.638240 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:15.639880 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:15.640699 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:15.642592 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:15.643343 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:15.643495 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:15.647114 (+ 3619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:15.647121 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:15.647128 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:15.647167 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:15.650051 (+ 2884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:15.650922 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:15.650928 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:15.650932 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:15.651072 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:15.651602 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:15.651605 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:15.651608 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:15.651926 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:15.652087 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:15.652905 (+ 818us) server_negotiation.cc:299] Negotiation successful
0810 18:45:15.653152 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":366,"thread_start_us":253,"threads_started":1}
I20260810 18:45:15.654621 26729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:15.635209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.193:55605)
0810 18:45:15.635917 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:15.635957 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:15.637707 (+ 1750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:15.638560 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:15.638571 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:15.639102 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:45:15.639690 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:15.639708 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:15.640858 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:15.640871 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:15.642256 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:15.642274 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:15.643611 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:15.643621 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:15.643844 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:15.644561 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:45:15.644592 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:45:15.646887 (+ 2295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:45:15.650213 (+ 3326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:15.650222 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:15.650239 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:15.650784 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:15.651190 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:15.651195 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:15.651199 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:15.651459 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:15.652063 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:45:15.652071 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:45:15.652472 (+ 401us) client_negotiation.cc:769] Sending connection context
0810 18:45:15.652787 (+ 315us) client_negotiation.cc:240] Negotiation successful
0810 18:45:15.653367 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":277,"threads_started":1}
I20260810 18:45:15.656638 26723 heartbeater.cc:347] Connected to a master server at 127.4.228.254:45083
I20260810 18:45:15.656975 26723 heartbeater.cc:464] Registering TS with master...
I20260810 18:45:15.657752 26723 heartbeater.cc:511] Master 127.4.228.254:45083 requested a full tablet report, sending...
I20260810 18:45:15.660012 26450 ts_manager.cc:194] Registered new tserver with Master: 6114ee077fd140739ff7a63019042a8f (127.4.228.193:42335)
I20260810 18:45:15.662132 26450 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.193@KRBTEST.COM'} at 127.4.228.193:55605
WARNING: no policy specified for kudu/127.4.228.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab.
Entry for principal kudu/127.4.228.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab.
I20260810 18:45:15.710902 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.194:0
--local_ip_for_outbound_sockets=127.4.228.194
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:45083
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:40315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:45:16.071108 26733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:45:16.071396 26733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:45:16.071491 26733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:45:16.078711 26733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:45:16.078819 26733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:45:16.078928 26733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.194
I20260810 18:45:16.088359 26733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:40315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:45083
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.194
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:45:16.089632 26733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:45:16.091046 26733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:45:16.105028 26743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:16.105121 26745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:16.105096 26749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:45:16.105923 26733 server_base.cc:1034] running on GCE node
I20260810 18:45:16.106681 26733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:45:16.107637 26733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:45:16.108896 26733 hybrid_clock.cc:648] HybridClock initialized: now 1786387516108847 us; error 30 us; skew 500 ppm
I20260810 18:45:16.109268 26733 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:45:16 dist-test-slave-7kzw krb5kdc[26383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387516, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:45:16.112720 26733 init.cc:377] Logged in from keytab as kudu/127.4.228.194@KRBTEST.COM (short username kudu)
I20260810 18:45:16.114286 26733 webserver.cc:460] Webserver started at http://127.4.228.194:44003/ using document root <none> and password file <none>
I20260810 18:45:16.115104 26733 fs_manager.cc:359] Metadata directory not provided
I20260810 18:45:16.115242 26733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:45:16.115574 26733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:45:16.118613 26733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/data/instance:
uuid: "108755ecae3544b8b7470da9de6965a6"
format_stamp: "Formatted at 2026-08-10 18:45:16 on dist-test-slave-7kzw"
server_key: "f05f8e081b4dc5adb1d2d5e8e46427a3"
server_key_iv: "2808b3b9383d60d98a14995237c418a6"
server_key_version: "encryptionkey@0"
I20260810 18:45:16.119401 26733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance:
uuid: "108755ecae3544b8b7470da9de6965a6"
format_stamp: "Formatted at 2026-08-10 18:45:16 on dist-test-slave-7kzw"
server_key: "f05f8e081b4dc5adb1d2d5e8e46427a3"
server_key_iv: "2808b3b9383d60d98a14995237c418a6"
server_key_version: "encryptionkey@0"
I20260810 18:45:16.124815 26733 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260810 18:45:16.128449 26760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:16.129930 26733 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:45:16.130124 26733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/wal
uuid: "108755ecae3544b8b7470da9de6965a6"
format_stamp: "Formatted at 2026-08-10 18:45:16 on dist-test-slave-7kzw"
server_key: "f05f8e081b4dc5adb1d2d5e8e46427a3"
server_key_iv: "2808b3b9383d60d98a14995237c418a6"
server_key_version: "encryptionkey@0"
I20260810 18:45:16.130339 26733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:45:16.154413 26733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:45:16.159142 26733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:45:16.159479 26733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:45:16.160880 26733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:45:16.163571 26733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:45:16.163662 26733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:16.163813 26733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:45:16.163864 26733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:16.199899 26733 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.194:44971
I20260810 18:45:16.199967 26922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.194:44971 every 8 connection(s)
I20260810 18:45:16.201333 26733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
I20260810 18:45:16.211490 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 26733
I20260810 18:45:16.211747 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance
I20260810 18:45:16.212256 5011 external_mini_cluster.cc:1428] Setting key da75a4223167ef879bf8ffc2ce4e0d89
Aug 10 18:45:16 dist-test-slave-7kzw krb5kdc[26383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387516, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:45:16.224479 26929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:16.204340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:45275 (local address 127.4.228.254:45083)
0810 18:45:16.204904 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:16.204913 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:16.206067 (+ 1154us) server_negotiation.cc:407] Connection header received
0810 18:45:16.208100 (+ 2033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:16.208105 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:16.208183 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:16.208309 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:16.209900 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:16.210776 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:16.212548 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:16.213148 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:16.213295 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:16.217985 (+ 4690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:16.217991 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:16.217995 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:16.218029 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:16.220729 (+ 2700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:16.221794 (+ 1065us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:16.221799 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:16.221801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:16.221951 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:16.222582 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:16.222586 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:16.222588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:16.222896 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:16.223025 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:16.223991 (+ 966us) server_negotiation.cc:299] Negotiation successful
0810 18:45:16.224192 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":291,"threads_started":1}
I20260810 18:45:16.226146 26930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:16.205025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.194:45275)
0810 18:45:16.205877 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:16.205927 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:16.207812 (+ 1885us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:16.208524 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:16.208533 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:16.209089 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:45:16.209702 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:16.209721 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:16.210948 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:16.210953 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:16.212370 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:16.212389 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:16.213436 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:16.213447 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:16.213802 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:16.214719 (+ 917us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:45:16.214753 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:45:16.217682 (+ 2929us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:45:16.220891 (+ 3209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:16.220904 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:16.220926 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:16.221646 (+ 720us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:16.222078 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:16.222085 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:16.222092 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:16.222363 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:16.223048 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:45:16.223059 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:45:16.223551 (+ 492us) client_negotiation.cc:769] Sending connection context
0810 18:45:16.223904 (+ 353us) client_negotiation.cc:240] Negotiation successful
0810 18:45:16.224517 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":341,"threads_started":1}
I20260810 18:45:16.227860 26927 heartbeater.cc:347] Connected to a master server at 127.4.228.254:45083
I20260810 18:45:16.228221 26927 heartbeater.cc:464] Registering TS with master...
I20260810 18:45:16.229007 26927 heartbeater.cc:511] Master 127.4.228.254:45083 requested a full tablet report, sending...
I20260810 18:45:16.230762 26450 ts_manager.cc:194] Registered new tserver with Master: 108755ecae3544b8b7470da9de6965a6 (127.4.228.194:44971)
I20260810 18:45:16.232080 26450 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.194@KRBTEST.COM'} at 127.4.228.194:45275
WARNING: no policy specified for kudu/127.4.228.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab.
Entry for principal kudu/127.4.228.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab.
I20260810 18:45:16.282826 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.195:0
--local_ip_for_outbound_sockets=127.4.228.195
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:45083
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:40315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:45:16.636248 26934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:45:16.636533 26934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:45:16.636714 26934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:45:16.644532 26934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:45:16.644665 26934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:45:16.644861 26934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.195
I20260810 18:45:16.655426 26934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:40315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:45083
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.195
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:45:16.656881 26934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:45:16.658598 26934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 18:45:16.666662 26723 heartbeater.cc:503] Master 127.4.228.254:45083 was elected leader, sending a full tablet report...
W20260810 18:45:16.673245 26945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:16.673354 26942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:16.673493 26941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:45:16.673977 26934 server_base.cc:1034] running on GCE node
I20260810 18:45:16.674811 26934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:45:16.676008 26934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:45:16.677274 26934 hybrid_clock.cc:648] HybridClock initialized: now 1786387516677225 us; error 54 us; skew 500 ppm
I20260810 18:45:16.677660 26934 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:45:16 dist-test-slave-7kzw krb5kdc[26383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387516, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:45:16.681308 26934 init.cc:377] Logged in from keytab as kudu/127.4.228.195@KRBTEST.COM (short username kudu)
I20260810 18:45:16.683066 26934 webserver.cc:460] Webserver started at http://127.4.228.195:42761/ using document root <none> and password file <none>
I20260810 18:45:16.683822 26934 fs_manager.cc:359] Metadata directory not provided
I20260810 18:45:16.683979 26934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:45:16.684324 26934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:45:16.687505 26934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/data/instance:
uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd"
format_stamp: "Formatted at 2026-08-10 18:45:16 on dist-test-slave-7kzw"
server_key: "fc452c4f5a67070abd032841211552a2"
server_key_iv: "79d0d0440dd80db90db48c342a8de42b"
server_key_version: "encryptionkey@0"
I20260810 18:45:16.688299 26934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance:
uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd"
format_stamp: "Formatted at 2026-08-10 18:45:16 on dist-test-slave-7kzw"
server_key: "fc452c4f5a67070abd032841211552a2"
server_key_iv: "79d0d0440dd80db90db48c342a8de42b"
server_key_version: "encryptionkey@0"
I20260810 18:45:16.693917 26934 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260810 18:45:16.697774 26953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:16.699374 26934 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:45:16.699565 26934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/wal
uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd"
format_stamp: "Formatted at 2026-08-10 18:45:16 on dist-test-slave-7kzw"
server_key: "fc452c4f5a67070abd032841211552a2"
server_key_iv: "79d0d0440dd80db90db48c342a8de42b"
server_key_version: "encryptionkey@0"
I20260810 18:45:16.699779 26934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:45:16.711892 26934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:45:16.716311 26934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:45:16.716631 26934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:45:16.717650 26934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:45:16.719340 26934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:45:16.719451 26934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:16.719609 26934 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:45:16.719692 26934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:16.756987 26934 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.195:38771
I20260810 18:45:16.757112 27119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.195:38771 every 8 connection(s)
I20260810 18:45:16.758622 26934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
I20260810 18:45:16.763545 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 26934
I20260810 18:45:16.763770 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance
I20260810 18:45:16.764417 5011 external_mini_cluster.cc:1428] Setting key d66f0665704d2d209729026b0b3f7888
Aug 10 18:45:16 dist-test-slave-7kzw krb5kdc[26383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387516, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:45:16.780443 27125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:16.761499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:40997 (local address 127.4.228.254:45083)
0810 18:45:16.762086 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:16.762091 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:16.762861 (+ 770us) server_negotiation.cc:407] Connection header received
0810 18:45:16.765489 (+ 2628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:16.765494 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:16.765638 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:16.765794 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:16.767729 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:16.768586 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:16.770146 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:16.770624 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:16.770736 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:16.774233 (+ 3497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:16.774238 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:16.774242 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:16.774303 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:16.776963 (+ 2660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:16.777688 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:16.777692 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:16.777694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:16.777824 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:16.778278 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:16.778282 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:16.778284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:16.778679 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:16.778856 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:16.779798 (+ 942us) server_negotiation.cc:299] Negotiation successful
0810 18:45:16.780025 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":350,"threads_started":1}
I20260810 18:45:16.781544 27126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:16.761854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.195:40997)
0810 18:45:16.762680 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:16.762733 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:16.765139 (+ 2406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:16.766073 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:16.766085 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:16.766858 (+ 773us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:45:16.767495 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:16.767513 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:16.768735 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:16.768740 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:16.769962 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:16.769979 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:16.770848 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:16.770854 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:16.771053 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:16.771761 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:45:16.771784 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:45:16.774026 (+ 2242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:45:16.777108 (+ 3082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:16.777115 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:16.777129 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:16.777581 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:16.777924 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:16.777927 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:16.777929 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:16.778121 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:16.778857 (+ 736us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:45:16.778865 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:45:16.779318 (+ 453us) client_negotiation.cc:769] Sending connection context
0810 18:45:16.779668 (+ 350us) client_negotiation.cc:240] Negotiation successful
0810 18:45:16.780213 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":314,"threads_started":1}
I20260810 18:45:16.783092 27120 heartbeater.cc:347] Connected to a master server at 127.4.228.254:45083
I20260810 18:45:16.783445 27120 heartbeater.cc:464] Registering TS with master...
I20260810 18:45:16.784255 27120 heartbeater.cc:511] Master 127.4.228.254:45083 requested a full tablet report, sending...
I20260810 18:45:16.785957 26450 ts_manager.cc:194] Registered new tserver with Master: a8e4f6f0df3e4d3c8890b77a53cce6dd (127.4.228.195:38771)
I20260810 18:45:16.786854 26450 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.195@KRBTEST.COM'} at 127.4.228.195:40997
I20260810 18:45:16.793237 5011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:45:16.812119 27125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:16.798235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50576 (local address 127.4.228.254:45083)
0810 18:45:16.798397 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:16.798402 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:16.798988 (+ 586us) server_negotiation.cc:407] Connection header received
0810 18:45:16.799124 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:16.799130 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:16.799214 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:16.799305 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:16.800785 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:16.801526 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:16.803005 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:16.803430 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:16.803529 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:16.806283 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:16.806289 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:16.806293 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:16.806331 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:16.809176 (+ 2845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:16.809926 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:16.809932 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:16.809934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:16.810104 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:16.810624 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:16.810628 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:16.810630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:16.811015 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:16.811217 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:16.811672 (+ 455us) server_negotiation.cc:299] Negotiation successful
0810 18:45:16.811883 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 18:45:16.817283 26450 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50576:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:45:16.824415 26450 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:45:16.847762 27138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:16.837966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54252 (local address 127.4.228.194:44971)
0810 18:45:16.838937 (+ 971us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:16.838949 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:16.839662 (+ 713us) server_negotiation.cc:407] Connection header received
0810 18:45:16.840084 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:16.840095 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:16.840414 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:16.840619 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:45:16.841729 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:16.842925 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:16.844819 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:16.846342 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:16.846509 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:16.846735 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:16.847291 (+ 556us) server_negotiation.cc:299] Negotiation successful
0810 18:45:16.847452 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":789,"thread_start_us":1000,"threads_started":1}
I20260810 18:45:16.848054 27139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:16.837748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:44971 (local address 127.0.0.1:54252)
0810 18:45:16.839480 (+ 1732us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:16.839522 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:16.839784 (+ 262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:16.840936 (+ 1152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:16.840943 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:16.840976 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:45:16.841490 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:16.841504 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:16.843086 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:16.843090 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:16.844619 (+ 1529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:16.844638 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:16.846621 (+ 1983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:16.846631 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:16.846953 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:16.847067 (+ 114us) client_negotiation.cc:769] Sending connection context
0810 18:45:16.847217 (+ 150us) client_negotiation.cc:240] Negotiation successful
0810 18:45:16.847467 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1430,"mutex_wait_us":30,"thread_start_us":1887,"threads_started":1}
I20260810 18:45:16.853372 26829 tablet_service.cc:1467] Processing CreateTablet for tablet 10fb876926b942c099929f1bd01e5606 (DEFAULT_TABLE table=test-table [id=dafc42da5573446eb8f1316b2879f1db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:45:16.854923 27143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:16.840020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:42335 (local address 127.0.0.1:40988)
0810 18:45:16.840612 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:16.840630 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:16.840746 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:16.846872 (+ 6126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:16.846880 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:16.846914 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:45:16.847544 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:16.847557 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:16.850177 (+ 2620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:16.850184 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:16.852270 (+ 2086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:16.852292 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:16.853632 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:16.853638 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:16.854185 (+ 547us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:16.854232 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 18:45:16.854324 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 18:45:16.854518 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"mutex_wait_us":196,"thread_start_us":291,"threads_started":1}
I20260810 18:45:16.855724 26829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10fb876926b942c099929f1bd01e5606. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:45:16.855751 27144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:16.840412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40988 (local address 127.4.228.193:42335)
0810 18:45:16.846127 (+ 5715us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:16.846136 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:16.846166 (+ 30us) server_negotiation.cc:407] Connection header received
0810 18:45:16.846265 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:16.846274 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:16.846591 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:16.847089 (+ 498us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:45:16.847769 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:16.850024 (+ 2255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:16.852454 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:16.853407 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:16.853578 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:16.853828 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:16.855281 (+ 1453us) server_negotiation.cc:299] Negotiation successful
0810 18:45:16.855437 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5528,"thread_start_us":5375,"threads_started":1}
I20260810 18:45:16.856632 27137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:16.838620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:38771 (local address 127.0.0.1:40758)
0810 18:45:16.839599 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:16.839618 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:16.839789 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:16.848903 (+ 9114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:16.848908 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:16.848944 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:45:16.849400 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:16.849414 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:16.851776 (+ 2362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:16.851783 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:16.853939 (+ 2156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:16.853956 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:16.855780 (+ 1824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:16.855789 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:16.855982 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:16.856020 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 18:45:16.856129 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 18:45:16.856263 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":849,"mutex_wait_us":27,"thread_start_us":438,"threads_started":1}
I20260810 18:45:16.856709 27140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:16.839330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40758 (local address 127.4.228.195:38771)
0810 18:45:16.847873 (+ 8543us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:16.847883 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:16.847924 (+ 41us) server_negotiation.cc:407] Connection header received
0810 18:45:16.848036 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:16.848046 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:16.848653 (+ 607us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:16.849584 (+ 931us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:45:16.849905 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:16.851611 (+ 1706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:16.854150 (+ 2539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:16.855561 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:16.855708 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:16.855946 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:16.856186 (+ 240us) server_negotiation.cc:299] Negotiation successful
0810 18:45:16.856367 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7790,"thread_start_us":7611,"threads_started":1}
I20260810 18:45:16.859946 26630 tablet_service.cc:1467] Processing CreateTablet for tablet 10fb876926b942c099929f1bd01e5606 (DEFAULT_TABLE table=test-table [id=dafc42da5573446eb8f1316b2879f1db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:45:16.860422 27019 tablet_service.cc:1467] Processing CreateTablet for tablet 10fb876926b942c099929f1bd01e5606 (DEFAULT_TABLE table=test-table [id=dafc42da5573446eb8f1316b2879f1db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:45:16.861569 26630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10fb876926b942c099929f1bd01e5606. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:45:16.861821 27019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10fb876926b942c099929f1bd01e5606. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:45:16.869966 27146 tablet_bootstrap.cc:492] T 10fb876926b942c099929f1bd01e5606 P 108755ecae3544b8b7470da9de6965a6: Bootstrap starting.
I20260810 18:45:16.873407 27146 tablet_bootstrap.cc:654] T 10fb876926b942c099929f1bd01e5606 P 108755ecae3544b8b7470da9de6965a6: Neither blocks nor log segments found. Creating new log.
I20260810 18:45:16.874667 27146 log.cc:824] T 10fb876926b942c099929f1bd01e5606 P 108755ecae3544b8b7470da9de6965a6: Log is configured to *not* fsync() on all Append() calls
I20260810 18:45:16.877228 27149 tablet_bootstrap.cc:492] T 10fb876926b942c099929f1bd01e5606 P 6114ee077fd140739ff7a63019042a8f: Bootstrap starting.
I20260810 18:45:16.880175 27146 tablet_bootstrap.cc:492] T 10fb876926b942c099929f1bd01e5606 P 108755ecae3544b8b7470da9de6965a6: No bootstrap required, opened a new log
I20260810 18:45:16.880522 27146 ts_tablet_manager.cc:1397] T 10fb876926b942c099929f1bd01e5606 P 108755ecae3544b8b7470da9de6965a6: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20260810 18:45:16.881250 27150 tablet_bootstrap.cc:492] T 10fb876926b942c099929f1bd01e5606 P a8e4f6f0df3e4d3c8890b77a53cce6dd: Bootstrap starting.
I20260810 18:45:16.881587 27149 tablet_bootstrap.cc:654] T 10fb876926b942c099929f1bd01e5606 P 6114ee077fd140739ff7a63019042a8f: Neither blocks nor log segments found. Creating new log.
I20260810 18:45:16.883311 27149 log.cc:824] T 10fb876926b942c099929f1bd01e5606 P 6114ee077fd140739ff7a63019042a8f: Log is configured to *not* fsync() on all Append() calls
I20260810 18:45:16.884729 27150 tablet_bootstrap.cc:654] T 10fb876926b942c099929f1bd01e5606 P a8e4f6f0df3e4d3c8890b77a53cce6dd: Neither blocks nor log segments found. Creating new log.
I20260810 18:45:16.885757 27150 log.cc:824] T 10fb876926b942c099929f1bd01e5606 P a8e4f6f0df3e4d3c8890b77a53cce6dd: Log is configured to *not* fsync() on all Append() calls
I20260810 18:45:16.886904 27149 tablet_bootstrap.cc:492] T 10fb876926b942c099929f1bd01e5606 P 6114ee077fd140739ff7a63019042a8f: No bootstrap required, opened a new log
I20260810 18:45:16.887195 27149 ts_tablet_manager.cc:1397] T 10fb876926b942c099929f1bd01e5606 P 6114ee077fd140739ff7a63019042a8f: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260810 18:45:16.888795 27150 tablet_bootstrap.cc:492] T 10fb876926b942c099929f1bd01e5606 P a8e4f6f0df3e4d3c8890b77a53cce6dd: No bootstrap required, opened a new log
I20260810 18:45:16.889075 27150 ts_tablet_manager.cc:1397] T 10fb876926b942c099929f1bd01e5606 P a8e4f6f0df3e4d3c8890b77a53cce6dd: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260810 18:45:16.892876 27146 raft_consensus.cc:348] T 10fb876926b942c099929f1bd01e5606 P 108755ecae3544b8b7470da9de6965a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108755ecae3544b8b7470da9de6965a6" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44971 } } peers { permanent_uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38771 } } peers { permanent_uuid: "6114ee077fd140739ff7a63019042a8f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42335 } }
I20260810 18:45:16.893388 27146 raft_consensus.cc:374] T 10fb876926b942c099929f1bd01e5606 P 108755ecae3544b8b7470da9de6965a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:45:16.893498 27146 raft_consensus.cc:729] T 10fb876926b942c099929f1bd01e5606 P 108755ecae3544b8b7470da9de6965a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 108755ecae3544b8b7470da9de6965a6, State: Initialized, Role: FOLLOWER
I20260810 18:45:16.894193 27146 consensus_queue.cc:260] T 10fb876926b942c099929f1bd01e5606 P 108755ecae3544b8b7470da9de6965a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108755ecae3544b8b7470da9de6965a6" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44971 } } peers { permanent_uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38771 } } peers { permanent_uuid: "6114ee077fd140739ff7a63019042a8f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42335 } }
I20260810 18:45:16.897104 27150 raft_consensus.cc:348] T 10fb876926b942c099929f1bd01e5606 P a8e4f6f0df3e4d3c8890b77a53cce6dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108755ecae3544b8b7470da9de6965a6" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44971 } } peers { permanent_uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38771 } } peers { permanent_uuid: "6114ee077fd140739ff7a63019042a8f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42335 } }
I20260810 18:45:16.897579 27150 raft_consensus.cc:374] T 10fb876926b942c099929f1bd01e5606 P a8e4f6f0df3e4d3c8890b77a53cce6dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:45:16.897704 27150 raft_consensus.cc:729] T 10fb876926b942c099929f1bd01e5606 P a8e4f6f0df3e4d3c8890b77a53cce6dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8e4f6f0df3e4d3c8890b77a53cce6dd, State: Initialized, Role: FOLLOWER
I20260810 18:45:16.898411 27150 consensus_queue.cc:260] T 10fb876926b942c099929f1bd01e5606 P a8e4f6f0df3e4d3c8890b77a53cce6dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108755ecae3544b8b7470da9de6965a6" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44971 } } peers { permanent_uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38771 } } peers { permanent_uuid: "6114ee077fd140739ff7a63019042a8f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42335 } }
I20260810 18:45:16.899367 26927 heartbeater.cc:503] Master 127.4.228.254:45083 was elected leader, sending a full tablet report...
I20260810 18:45:16.899415 27149 raft_consensus.cc:348] T 10fb876926b942c099929f1bd01e5606 P 6114ee077fd140739ff7a63019042a8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108755ecae3544b8b7470da9de6965a6" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44971 } } peers { permanent_uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38771 } } peers { permanent_uuid: "6114ee077fd140739ff7a63019042a8f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42335 } }
I20260810 18:45:16.899887 27149 raft_consensus.cc:374] T 10fb876926b942c099929f1bd01e5606 P 6114ee077fd140739ff7a63019042a8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:45:16.900003 27149 raft_consensus.cc:729] T 10fb876926b942c099929f1bd01e5606 P 6114ee077fd140739ff7a63019042a8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6114ee077fd140739ff7a63019042a8f, State: Initialized, Role: FOLLOWER
I20260810 18:45:16.900146 27146 ts_tablet_manager.cc:1428] T 10fb876926b942c099929f1bd01e5606 P 108755ecae3544b8b7470da9de6965a6: Time spent starting tablet: real 0.019s user 0.012s sys 0.007s
I20260810 18:45:16.900663 27149 consensus_queue.cc:260] T 10fb876926b942c099929f1bd01e5606 P 6114ee077fd140739ff7a63019042a8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108755ecae3544b8b7470da9de6965a6" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44971 } } peers { permanent_uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38771 } } peers { permanent_uuid: "6114ee077fd140739ff7a63019042a8f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42335 } }
I20260810 18:45:16.901961 27150 ts_tablet_manager.cc:1428] T 10fb876926b942c099929f1bd01e5606 P a8e4f6f0df3e4d3c8890b77a53cce6dd: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260810 18:45:16.902529 27120 heartbeater.cc:503] Master 127.4.228.254:45083 was elected leader, sending a full tablet report...
I20260810 18:45:16.906364 27149 ts_tablet_manager.cc:1428] T 10fb876926b942c099929f1bd01e5606 P 6114ee077fd140739ff7a63019042a8f: Time spent starting tablet: real 0.019s user 0.011s sys 0.008s
I20260810 18:45:16.937776 27158 raft_consensus.cc:482] T 10fb876926b942c099929f1bd01e5606 P a8e4f6f0df3e4d3c8890b77a53cce6dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:45:16.938211 27158 raft_consensus.cc:504] T 10fb876926b942c099929f1bd01e5606 P a8e4f6f0df3e4d3c8890b77a53cce6dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108755ecae3544b8b7470da9de6965a6" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44971 } } peers { permanent_uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38771 } } peers { permanent_uuid: "6114ee077fd140739ff7a63019042a8f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42335 } }
I20260810 18:45:16.940224 27158 leader_election.cc:288] T 10fb876926b942c099929f1bd01e5606 P a8e4f6f0df3e4d3c8890b77a53cce6dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 108755ecae3544b8b7470da9de6965a6 (127.4.228.194:44971), 6114ee077fd140739ff7a63019042a8f (127.4.228.193:42335)
I20260810 18:45:16.947817 27138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:16.940739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:44253 (local address 127.4.228.194:44971)
0810 18:45:16.941000 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:16.941007 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:16.941032 (+ 25us) server_negotiation.cc:407] Connection header received
0810 18:45:16.941116 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:16.941125 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:16.941251 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:16.941380 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:45:16.942013 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:16.943054 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:16.945480 (+ 2426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:16.946849 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:16.947034 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:16.947122 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:16.947394 (+ 272us) server_negotiation.cc:299] Negotiation successful
0810 18:45:16.947548 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260810 18:45:16.947826 27126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:16.940542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:44971 (local address 127.4.228.195:44253)
0810 18:45:16.940765 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:16.940795 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:16.940949 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:16.941528 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:16.941533 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:16.941564 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:45:16.941802 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:16.941815 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:16.943222 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:16.943227 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:16.945281 (+ 2054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:16.945297 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:16.947039 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:16.947044 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:16.947196 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:16.947222 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 18:45:16.947325 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 18:45:16.947422 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":1}
I20260810 18:45:16.948372 27144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:16.941349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:34697 (local address 127.4.228.193:42335)
0810 18:45:16.941682 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:16.941687 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:16.941807 (+ 120us) server_negotiation.cc:407] Connection header received
0810 18:45:16.942026 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:16.942032 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:16.942117 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:16.942227 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:45:16.942824 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:16.943659 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:16.945930 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:16.947370 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:16.947547 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:16.947622 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:16.948015 (+ 393us) server_negotiation.cc:299] Negotiation successful
0810 18:45:16.948192 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176}
I20260810 18:45:16.948933 27160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:16.940995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:42335 (local address 127.4.228.195:34697)
0810 18:45:16.941688 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:16.941712 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:16.941837 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:16.942252 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:16.942255 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:16.942276 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:45:16.942668 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:16.942680 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:16.943850 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:16.943856 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:16.945753 (+ 1897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:16.945764 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:16.947611 (+ 1847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:16.947617 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:16.947792 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:16.947836 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 18:45:16.948532 (+ 696us) client_negotiation.cc:240] Negotiation successful
0810 18:45:16.948686 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":336,"threads_started":1}
I20260810 18:45:16.948983 26861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10fb876926b942c099929f1bd01e5606" candidate_uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "108755ecae3544b8b7470da9de6965a6" is_pre_election: true
I20260810 18:45:16.949656 26861 raft_consensus.cc:2454] T 10fb876926b942c099929f1bd01e5606 P 108755ecae3544b8b7470da9de6965a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8e4f6f0df3e4d3c8890b77a53cce6dd in term 0.
I20260810 18:45:16.950107 26663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10fb876926b942c099929f1bd01e5606" candidate_uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6114ee077fd140739ff7a63019042a8f" is_pre_election: true
I20260810 18:45:16.950450 26954 leader_election.cc:302] T 10fb876926b942c099929f1bd01e5606 P a8e4f6f0df3e4d3c8890b77a53cce6dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 108755ecae3544b8b7470da9de6965a6, a8e4f6f0df3e4d3c8890b77a53cce6dd; no voters:
I20260810 18:45:16.950825 26663 raft_consensus.cc:2454] T 10fb876926b942c099929f1bd01e5606 P 6114ee077fd140739ff7a63019042a8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8e4f6f0df3e4d3c8890b77a53cce6dd in term 0.
I20260810 18:45:16.950958 27158 raft_consensus.cc:2781] T 10fb876926b942c099929f1bd01e5606 P a8e4f6f0df3e4d3c8890b77a53cce6dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:45:16.951128 27158 raft_consensus.cc:482] T 10fb876926b942c099929f1bd01e5606 P a8e4f6f0df3e4d3c8890b77a53cce6dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:45:16.951237 27158 raft_consensus.cc:3037] T 10fb876926b942c099929f1bd01e5606 P a8e4f6f0df3e4d3c8890b77a53cce6dd [term 0 FOLLOWER]: Advancing to term 1
W20260810 18:45:16.953517 26928 tablet.cc:2369] T 10fb876926b942c099929f1bd01e5606 P 108755ecae3544b8b7470da9de6965a6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:45:16.954619 27158 raft_consensus.cc:504] T 10fb876926b942c099929f1bd01e5606 P a8e4f6f0df3e4d3c8890b77a53cce6dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108755ecae3544b8b7470da9de6965a6" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44971 } } peers { permanent_uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38771 } } peers { permanent_uuid: "6114ee077fd140739ff7a63019042a8f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42335 } }
I20260810 18:45:16.955426 27158 leader_election.cc:288] T 10fb876926b942c099929f1bd01e5606 P a8e4f6f0df3e4d3c8890b77a53cce6dd [CANDIDATE]: Term 1 election: Requested vote from peers 108755ecae3544b8b7470da9de6965a6 (127.4.228.194:44971), 6114ee077fd140739ff7a63019042a8f (127.4.228.193:42335)
I20260810 18:45:16.955869 26861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10fb876926b942c099929f1bd01e5606" candidate_uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "108755ecae3544b8b7470da9de6965a6"
I20260810 18:45:16.955940 26663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10fb876926b942c099929f1bd01e5606" candidate_uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6114ee077fd140739ff7a63019042a8f"
I20260810 18:45:16.956079 26861 raft_consensus.cc:3037] T 10fb876926b942c099929f1bd01e5606 P 108755ecae3544b8b7470da9de6965a6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:45:16.956231 26663 raft_consensus.cc:3037] T 10fb876926b942c099929f1bd01e5606 P 6114ee077fd140739ff7a63019042a8f [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:45:16.958446 26663 raft_consensus.cc:2454] T 10fb876926b942c099929f1bd01e5606 P 6114ee077fd140739ff7a63019042a8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8e4f6f0df3e4d3c8890b77a53cce6dd in term 1.
I20260810 18:45:16.958935 26954 leader_election.cc:302] T 10fb876926b942c099929f1bd01e5606 P a8e4f6f0df3e4d3c8890b77a53cce6dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6114ee077fd140739ff7a63019042a8f, a8e4f6f0df3e4d3c8890b77a53cce6dd; no voters:
I20260810 18:45:16.959234 27158 raft_consensus.cc:2781] T 10fb876926b942c099929f1bd01e5606 P a8e4f6f0df3e4d3c8890b77a53cce6dd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:45:16.959302 26861 raft_consensus.cc:2454] T 10fb876926b942c099929f1bd01e5606 P 108755ecae3544b8b7470da9de6965a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8e4f6f0df3e4d3c8890b77a53cce6dd in term 1.
I20260810 18:45:16.959761 27158 raft_consensus.cc:686] T 10fb876926b942c099929f1bd01e5606 P a8e4f6f0df3e4d3c8890b77a53cce6dd [term 1 LEADER]: Becoming Leader. State: Replica: a8e4f6f0df3e4d3c8890b77a53cce6dd, State: Running, Role: LEADER
I20260810 18:45:16.960285 27158 consensus_queue.cc:237] T 10fb876926b942c099929f1bd01e5606 P a8e4f6f0df3e4d3c8890b77a53cce6dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108755ecae3544b8b7470da9de6965a6" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44971 } } peers { permanent_uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38771 } } peers { permanent_uuid: "6114ee077fd140739ff7a63019042a8f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42335 } }
I20260810 18:45:16.965188 26446 catalog_manager.cc:5495] T 10fb876926b942c099929f1bd01e5606 P a8e4f6f0df3e4d3c8890b77a53cce6dd reported cstate change: term changed from 0 to 1, leader changed from <none> to a8e4f6f0df3e4d3c8890b77a53cce6dd (127.4.228.195). New cstate: current_term: 1 leader_uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108755ecae3544b8b7470da9de6965a6" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6114ee077fd140739ff7a63019042a8f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42335 } health_report { overall_health: UNKNOWN } } }
I20260810 18:45:17.008688 27140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:17.002421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40764 (local address 127.4.228.195:38771)
0810 18:45:17.002699 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:17.002706 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:17.002828 (+ 122us) server_negotiation.cc:407] Connection header received
0810 18:45:17.003098 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:17.003102 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:17.003277 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:17.003447 (+ 170us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:45:17.003936 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:17.004813 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:17.006292 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:17.006748 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:17.006892 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:17.007489 (+ 597us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:45:17.008031 (+ 542us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:45:17.008215 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:17.008324 (+ 109us) server_negotiation.cc:299] Negotiation successful
0810 18:45:17.008411 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
W20260810 18:45:17.010452 27123 tablet.cc:2369] T 10fb876926b942c099929f1bd01e5606 P a8e4f6f0df3e4d3c8890b77a53cce6dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:45:17.018132 26663 raft_consensus.cc:1261] T 10fb876926b942c099929f1bd01e5606 P 6114ee077fd140739ff7a63019042a8f [term 1 FOLLOWER]: Refusing update from remote peer a8e4f6f0df3e4d3c8890b77a53cce6dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:45:17.018144 26861 raft_consensus.cc:1261] T 10fb876926b942c099929f1bd01e5606 P 108755ecae3544b8b7470da9de6965a6 [term 1 FOLLOWER]: Refusing update from remote peer a8e4f6f0df3e4d3c8890b77a53cce6dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:45:17.019378 27163 consensus_queue.cc:1035] T 10fb876926b942c099929f1bd01e5606 P a8e4f6f0df3e4d3c8890b77a53cce6dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6114ee077fd140739ff7a63019042a8f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:45:17.019848 27158 consensus_queue.cc:1035] T 10fb876926b942c099929f1bd01e5606 P a8e4f6f0df3e4d3c8890b77a53cce6dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "108755ecae3544b8b7470da9de6965a6" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:45:17.058450 26446 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:45:17.058826 26446 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:45:17.063001 26446 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198: Sending DeleteTablet for 3 replicas of tablet 10fb876926b942c099929f1bd01e5606
I20260810 18:45:17.064316 27019 tablet_service.cc:1514] Processing DeleteTablet for tablet 10fb876926b942c099929f1bd01e5606 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:45:17 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:40758
I20260810 18:45:17.064371 26829 tablet_service.cc:1514] Processing DeleteTablet for tablet 10fb876926b942c099929f1bd01e5606 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:45:17 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:54252
I20260810 18:45:17.064518 26630 tablet_service.cc:1514] Processing DeleteTablet for tablet 10fb876926b942c099929f1bd01e5606 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:45:17 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:40988
I20260810 18:45:17.065687 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 26541
I20260810 18:45:17.076129 27184 tablet_replica.cc:330] T 10fb876926b942c099929f1bd01e5606 P 108755ecae3544b8b7470da9de6965a6: stopping tablet replica
I20260810 18:45:17.076427 27181 tablet_replica.cc:330] T 10fb876926b942c099929f1bd01e5606 P a8e4f6f0df3e4d3c8890b77a53cce6dd: stopping tablet replica
I20260810 18:45:17.076795 27184 raft_consensus.cc:2229] T 10fb876926b942c099929f1bd01e5606 P 108755ecae3544b8b7470da9de6965a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:45:17.077193 27181 raft_consensus.cc:2229] T 10fb876926b942c099929f1bd01e5606 P a8e4f6f0df3e4d3c8890b77a53cce6dd [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:45:17.077461 27184 raft_consensus.cc:2258] T 10fb876926b942c099929f1bd01e5606 P 108755ecae3544b8b7470da9de6965a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:45:17.078403 27181 raft_consensus.cc:2258] T 10fb876926b942c099929f1bd01e5606 P a8e4f6f0df3e4d3c8890b77a53cce6dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:45:17.080991 27181 ts_tablet_manager.cc:1905] T 10fb876926b942c099929f1bd01e5606 P a8e4f6f0df3e4d3c8890b77a53cce6dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:45:17.081202 27184 ts_tablet_manager.cc:1905] T 10fb876926b942c099929f1bd01e5606 P 108755ecae3544b8b7470da9de6965a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:45:17.087149 27182 tablet_replica.cc:330] T 10fb876926b942c099929f1bd01e5606 P 6114ee077fd140739ff7a63019042a8f: stopping tablet replica
I20260810 18:45:17.087757 27182 raft_consensus.cc:2229] T 10fb876926b942c099929f1bd01e5606 P 6114ee077fd140739ff7a63019042a8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:45:17.088452 27182 raft_consensus.cc:2258] T 10fb876926b942c099929f1bd01e5606 P 6114ee077fd140739ff7a63019042a8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:45:17.092444 27184 ts_tablet_manager.cc:1918] T 10fb876926b942c099929f1bd01e5606 P 108755ecae3544b8b7470da9de6965a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:45:17.092808 27181 ts_tablet_manager.cc:1918] T 10fb876926b942c099929f1bd01e5606 P a8e4f6f0df3e4d3c8890b77a53cce6dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:45:17.093001 27181 log.cc:1196] T 10fb876926b942c099929f1bd01e5606 P a8e4f6f0df3e4d3c8890b77a53cce6dd: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/wal/wals/10fb876926b942c099929f1bd01e5606
I20260810 18:45:17.093658 27181 ts_tablet_manager.cc:1939] T 10fb876926b942c099929f1bd01e5606 P a8e4f6f0df3e4d3c8890b77a53cce6dd: Deleting consensus metadata
I20260810 18:45:17.094969 27182 ts_tablet_manager.cc:1905] T 10fb876926b942c099929f1bd01e5606 P 6114ee077fd140739ff7a63019042a8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:45:17.096086 26430 catalog_manager.cc:4841] TS a8e4f6f0df3e4d3c8890b77a53cce6dd (127.4.228.195:38771): tablet 10fb876926b942c099929f1bd01e5606 (table test-table [id=dafc42da5573446eb8f1316b2879f1db]) successfully deleted
I20260810 18:45:17.096617 27184 log.cc:1196] T 10fb876926b942c099929f1bd01e5606 P 108755ecae3544b8b7470da9de6965a6: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/wal/wals/10fb876926b942c099929f1bd01e5606
I20260810 18:45:17.097399 27184 ts_tablet_manager.cc:1939] T 10fb876926b942c099929f1bd01e5606 P 108755ecae3544b8b7470da9de6965a6: Deleting consensus metadata
Aug 10 18:45:17 dist-test-slave-7kzw krb5kdc[26383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.193@KRBTEST.COM
I20260810 18:45:17.099642 26429 catalog_manager.cc:4841] TS 108755ecae3544b8b7470da9de6965a6 (127.4.228.194:44971): tablet 10fb876926b942c099929f1bd01e5606 (table test-table [id=dafc42da5573446eb8f1316b2879f1db]) successfully deleted
I20260810 18:45:17.102632 27182 ts_tablet_manager.cc:1918] T 10fb876926b942c099929f1bd01e5606 P 6114ee077fd140739ff7a63019042a8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:45:17.102839 27182 log.cc:1196] T 10fb876926b942c099929f1bd01e5606 P 6114ee077fd140739ff7a63019042a8f: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/wal/wals/10fb876926b942c099929f1bd01e5606
I20260810 18:45:17.103421 27182 ts_tablet_manager.cc:1939] T 10fb876926b942c099929f1bd01e5606 P 6114ee077fd140739ff7a63019042a8f: Deleting consensus metadata
I20260810 18:45:17.105209 26429 catalog_manager.cc:4841] TS 6114ee077fd140739ff7a63019042a8f (127.4.228.193:42335): tablet 10fb876926b942c099929f1bd01e5606 (table test-table [id=dafc42da5573446eb8f1316b2879f1db]) successfully deleted
I20260810 18:45:17.108980 27144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:17.066992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41004 (local address 127.4.228.193:42335)
0810 18:45:17.067186 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:17.067192 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:17.085997 (+ 18805us) server_negotiation.cc:407] Connection header received
0810 18:45:17.087234 (+ 1237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:17.087242 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:17.087361 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:17.087445 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:17.089711 (+ 2266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:17.090897 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:17.092939 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:17.093806 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:17.093920 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:17.100336 (+ 6416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:17.100354 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:17.100381 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:17.100452 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:17.105406 (+ 4954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:17.106180 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:17.106192 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:17.106198 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:17.106392 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:17.106923 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:17.106929 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:17.106933 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:17.107512 (+ 579us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:17.107848 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:17.108252 (+ 404us) server_negotiation.cc:299] Negotiation successful
0810 18:45:17.108512 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 18:45:17.109933 26714 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:45:17.207108 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 26541
I20260810 18:45:17.239447 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 26733
Aug 10 18:45:17 dist-test-slave-7kzw krb5kdc[26383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.194@KRBTEST.COM
I20260810 18:45:17.256263 27138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:17.240311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54260 (local address 127.4.228.194:44971)
0810 18:45:17.240485 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:17.240492 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:17.240567 (+ 75us) server_negotiation.cc:407] Connection header received
0810 18:45:17.240805 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:17.240811 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:17.240898 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:17.241011 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:17.242382 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:17.243313 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:17.244654 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:17.245058 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:17.245202 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:17.249778 (+ 4576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:17.249794 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:17.249812 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:17.249850 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:17.253186 (+ 3336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:17.253928 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:17.253944 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:17.253947 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:17.254088 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:17.254526 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:17.254532 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:17.254535 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:17.254978 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:17.255303 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:17.255665 (+ 362us) server_negotiation.cc:299] Negotiation successful
0810 18:45:17.255912 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 18:45:17.257020 26916 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:45:17.347740 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 26733
I20260810 18:45:17.375991 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 26934
Aug 10 18:45:17 dist-test-slave-7kzw krb5kdc[26383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.195@KRBTEST.COM
I20260810 18:45:17.396903 27140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:17.377004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40772 (local address 127.4.228.195:38771)
0810 18:45:17.377232 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:17.377239 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:17.377265 (+ 26us) server_negotiation.cc:407] Connection header received
0810 18:45:17.377412 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:17.377418 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:17.377536 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:17.377675 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:17.379349 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:17.380681 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:17.382148 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:17.383036 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:17.383176 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:17.388681 (+ 5505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:17.388705 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:17.388732 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:17.388794 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:17.393333 (+ 4539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:17.394156 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:17.394166 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:17.394170 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:17.394375 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:17.394936 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:17.394941 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:17.394945 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:17.395516 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:17.395832 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:17.396135 (+ 303us) server_negotiation.cc:299] Negotiation successful
0810 18:45:17.396389 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260810 18:45:17.397874 27115 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:45:17.526734 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 26934
I20260810 18:45:17.556167 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 26404
I20260810 18:45:17.557451 26494 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:45:17.676838 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 26404
I20260810 18:45:17.704991 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:45083
--webserver_interface=127.4.228.254
--webserver_port=33247
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:40315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:45083
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:45:18.060612 27195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:45:18.060948 27195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:45:18.061043 27195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:45:18.068547 27195 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:45:18.068673 27195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:45:18.068722 27195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:45:18.068773 27195 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:45:18.068820 27195 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:45:18.079241 27195 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:40315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:45083
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:45083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=33247
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:45:18.080639 27195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:45:18.081983 27195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:45:18.110594 27206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:18.110512 27207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:18.110545 27210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:45:18.111668 27195 server_base.cc:1034] running on GCE node
I20260810 18:45:18.112648 27195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:45:18.114377 27195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:45:18.115798 27195 hybrid_clock.cc:648] HybridClock initialized: now 1786387518115734 us; error 48 us; skew 500 ppm
I20260810 18:45:18.116330 27195 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:45:18 dist-test-slave-7kzw krb5kdc[26383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387518, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:45:18.120599 27195 init.cc:377] Logged in from keytab as kudu/127.4.228.254@KRBTEST.COM (short username kudu)
I20260810 18:45:18.122674 27195 webserver.cc:460] Webserver started at http://127.4.228.254:33247/ using document root <none> and password file <none>
I20260810 18:45:18.123548 27195 fs_manager.cc:359] Metadata directory not provided
I20260810 18:45:18.123694 27195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:45:18.130779 27195 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.003s
I20260810 18:45:18.134056 27217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:18.135699 27195 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:45:18.135914 27195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "346a38b450e0436191ddbba9e03b1198"
format_stamp: "Formatted at 2026-08-10 18:45:14 on dist-test-slave-7kzw"
server_key: "0776f4b220826e5549b8e15679d14529"
server_key_iv: "f27d06aa6bfe6f501895bb753c210983"
server_key_version: "encryptionkey@0"
I20260810 18:45:18.137058 27195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:45:18.154368 27195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:45:18.159916 27195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:45:18.160274 27195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:45:18.184636 27195 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:45083
I20260810 18:45:18.184639 27298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:45083 every 8 connection(s)
I20260810 18:45:18.186652 27195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:45:18.195070 27299 sys_catalog.cc:263] Verifying existing consensus state
I20260810 18:45:18.196278 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 27195
I20260810 18:45:18.197551 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.193:42335
--local_ip_for_outbound_sockets=127.4.228.193
--tserver_master_addrs=127.4.228.254:45083
--webserver_port=41535
--webserver_interface=127.4.228.193
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:40315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:45:18.201100 27299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198: Bootstrap starting.
I20260810 18:45:18.226687 27299 log.cc:824] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198: Log is configured to *not* fsync() on all Append() calls
I20260810 18:45:18.241467 27299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:45:18.242204 27299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198: Bootstrap complete.
I20260810 18:45:18.252992 27299 raft_consensus.cc:348] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "346a38b450e0436191ddbba9e03b1198" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45083 } }
I20260810 18:45:18.253988 27299 raft_consensus.cc:729] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 346a38b450e0436191ddbba9e03b1198, State: Initialized, Role: FOLLOWER
I20260810 18:45:18.254817 27299 consensus_queue.cc:260] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "346a38b450e0436191ddbba9e03b1198" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45083 } }
I20260810 18:45:18.255110 27299 raft_consensus.cc:388] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:45:18.255287 27299 raft_consensus.cc:482] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:45:18.255506 27299 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [term 1 FOLLOWER]: Advancing to term 2
I20260810 18:45:18.260011 27299 raft_consensus.cc:504] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "346a38b450e0436191ddbba9e03b1198" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45083 } }
I20260810 18:45:18.260667 27299 leader_election.cc:302] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 346a38b450e0436191ddbba9e03b1198; no voters:
I20260810 18:45:18.261423 27299 leader_election.cc:288] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 18:45:18.261605 27306 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 18:45:18.262917 27306 raft_consensus.cc:686] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [term 2 LEADER]: Becoming Leader. State: Replica: 346a38b450e0436191ddbba9e03b1198, State: Running, Role: LEADER
I20260810 18:45:18.263760 27306 consensus_queue.cc:237] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "346a38b450e0436191ddbba9e03b1198" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45083 } }
I20260810 18:45:18.264302 27299 sys_catalog.cc:564] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:45:18.266896 27309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 346a38b450e0436191ddbba9e03b1198. Latest consensus state: current_term: 2 leader_uuid: "346a38b450e0436191ddbba9e03b1198" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "346a38b450e0436191ddbba9e03b1198" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45083 } } }
I20260810 18:45:18.267442 27309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [sys.catalog]: This master's current role is: LEADER
I20260810 18:45:18.268016 27308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "346a38b450e0436191ddbba9e03b1198" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "346a38b450e0436191ddbba9e03b1198" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45083 } } }
I20260810 18:45:18.268396 27308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [sys.catalog]: This master's current role is: LEADER
I20260810 18:45:18.275907 27313 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:45:18.285794 27313 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:45:18.287778 27313 catalog_manager.cc:1259] Loaded cluster ID: 17650ec17c2247408e130a2363f61f0b
I20260810 18:45:18.287974 27313 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:45:18.293337 27313 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260810 18:45:18.319581 27313 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260810 18:45:18.341838 27313 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:45:18.343883 27313 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198: Loaded TSK: 0
I20260810 18:45:18.345649 27313 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 18:45:18.594365 27302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:45:18.594825 27302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:45:18.595042 27302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:45:18.602967 27302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:45:18.603145 27302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:45:18.603327 27302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.193
I20260810 18:45:18.613758 27302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:40315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.193:42335
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.228.193
--webserver_port=41535
--tserver_master_addrs=127.4.228.254:45083
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.193
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:45:18.615355 27302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:45:18.616885 27302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:45:18.632845 27348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:18.632838 27343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:18.633414 27342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:45:18.635245 27302 server_base.cc:1034] running on GCE node
I20260810 18:45:18.636161 27302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:45:18.637339 27302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:45:18.638711 27302 hybrid_clock.cc:648] HybridClock initialized: now 1786387518638636 us; error 59 us; skew 500 ppm
I20260810 18:45:18.639106 27302 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:45:18 dist-test-slave-7kzw krb5kdc[26383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387518, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:45:18.643100 27302 init.cc:377] Logged in from keytab as kudu/127.4.228.193@KRBTEST.COM (short username kudu)
I20260810 18:45:18.645047 27302 webserver.cc:460] Webserver started at http://127.4.228.193:41535/ using document root <none> and password file <none>
I20260810 18:45:18.645923 27302 fs_manager.cc:359] Metadata directory not provided
I20260810 18:45:18.646113 27302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:45:18.652657 27302 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:45:18.655885 27358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:18.657563 27302 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:45:18.657815 27302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/wal
uuid: "6114ee077fd140739ff7a63019042a8f"
format_stamp: "Formatted at 2026-08-10 18:45:15 on dist-test-slave-7kzw"
server_key: "030db39962bc74b50adf6bed4baf5176"
server_key_iv: "e4870105dfe0384426f9f00d9ed082d9"
server_key_version: "encryptionkey@0"
I20260810 18:45:18.659026 27302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:45:18.683215 27302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:45:18.688159 27302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:45:18.688508 27302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:45:18.689769 27302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:45:18.691452 27302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:45:18.691550 27302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:18.691720 27302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:45:18.691828 27302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:18.731117 27302 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.193:42335
I20260810 18:45:18.731218 27532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.193:42335 every 8 connection(s)
I20260810 18:45:18.732909 27302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
I20260810 18:45:18.743528 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 27302
I20260810 18:45:18.744472 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.194:44971
--local_ip_for_outbound_sockets=127.4.228.194
--tserver_master_addrs=127.4.228.254:45083
--webserver_port=44003
--webserver_interface=127.4.228.194
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:40315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:45:18 dist-test-slave-7kzw krb5kdc[26383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387518, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:45:18.762722 27536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:18.736585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.193:47909)
0810 18:45:18.737438 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:18.737487 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:18.739251 (+ 1764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:18.740799 (+ 1548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:18.740807 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:18.741428 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:45:18.742047 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:18.742064 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:18.744816 (+ 2752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:18.744825 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:18.747415 (+ 2590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:18.747437 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:18.748712 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:18.748723 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:18.749090 (+ 367us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:18.750154 (+ 1064us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:45:18.750196 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:45:18.753857 (+ 3661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:45:18.758137 (+ 4280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:18.758145 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:18.758159 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:18.758843 (+ 684us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:18.759409 (+ 566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:18.759413 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:18.759415 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:18.759604 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:18.760604 (+ 1000us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:45:18.760610 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:45:18.760920 (+ 310us) client_negotiation.cc:769] Sending connection context
0810 18:45:18.761153 (+ 233us) client_negotiation.cc:240] Negotiation successful
0810 18:45:18.761594 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":352,"threads_started":1}
I20260810 18:45:18.763233 27535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:18.736247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:47909 (local address 127.4.228.254:45083)
0810 18:45:18.736928 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:18.736948 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:18.737651 (+ 703us) server_negotiation.cc:407] Connection header received
0810 18:45:18.739861 (+ 2210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:18.739875 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:18.740249 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:18.740658 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:18.742313 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:18.744555 (+ 2242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:18.747711 (+ 3156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:18.748428 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:18.748637 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:18.754223 (+ 5586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:18.754233 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:18.754251 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:18.754292 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:18.757955 (+ 3663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:18.759016 (+ 1061us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:18.759029 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:18.759040 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:18.759286 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:18.759725 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:18.759733 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:18.759739 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:18.760461 (+ 722us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:18.760661 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:18.761468 (+ 807us) server_negotiation.cc:299] Negotiation successful
0810 18:45:18.761915 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":274,"threads_started":1}
I20260810 18:45:18.766140 27533 heartbeater.cc:347] Connected to a master server at 127.4.228.254:45083
I20260810 18:45:18.766523 27533 heartbeater.cc:464] Registering TS with master...
I20260810 18:45:18.767542 27533 heartbeater.cc:511] Master 127.4.228.254:45083 requested a full tablet report, sending...
I20260810 18:45:18.770334 27245 ts_manager.cc:194] Registered new tserver with Master: 6114ee077fd140739ff7a63019042a8f (127.4.228.193:42335)
I20260810 18:45:18.772058 27245 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.193@KRBTEST.COM'} at 127.4.228.193:47909
W20260810 18:45:19.105507 27537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:45:19.105850 27537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:45:19.106011 27537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:45:19.113593 27537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:45:19.113737 27537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:45:19.113909 27537 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.194
I20260810 18:45:19.124713 27537 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:40315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.194:44971
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.228.194
--webserver_port=44003
--tserver_master_addrs=127.4.228.254:45083
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.194
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:45:19.126307 27537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:45:19.127867 27537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:45:19.142637 27542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:19.143102 27544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:19.143796 27546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:45:19.144579 27537 server_base.cc:1034] running on GCE node
I20260810 18:45:19.145309 27537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:45:19.146529 27537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:45:19.147807 27537 hybrid_clock.cc:648] HybridClock initialized: now 1786387519147724 us; error 68 us; skew 500 ppm
I20260810 18:45:19.148186 27537 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:45:19 dist-test-slave-7kzw krb5kdc[26383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387519, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:45:19.152145 27537 init.cc:377] Logged in from keytab as kudu/127.4.228.194@KRBTEST.COM (short username kudu)
I20260810 18:45:19.153988 27537 webserver.cc:460] Webserver started at http://127.4.228.194:44003/ using document root <none> and password file <none>
I20260810 18:45:19.154942 27537 fs_manager.cc:359] Metadata directory not provided
I20260810 18:45:19.155151 27537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:45:19.161458 27537 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260810 18:45:19.165063 27558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:19.166769 27537 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 18:45:19.166972 27537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/wal
uuid: "108755ecae3544b8b7470da9de6965a6"
format_stamp: "Formatted at 2026-08-10 18:45:16 on dist-test-slave-7kzw"
server_key: "f05f8e081b4dc5adb1d2d5e8e46427a3"
server_key_iv: "2808b3b9383d60d98a14995237c418a6"
server_key_version: "encryptionkey@0"
I20260810 18:45:19.168021 27537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:45:19.236446 27537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:45:19.241339 27537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:45:19.241649 27537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:45:19.242800 27537 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:45:19.244261 27537 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:45:19.244364 27537 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:19.244545 27537 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:45:19.244630 27537 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:19.280194 27537 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.194:44971
I20260810 18:45:19.280212 27718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.194:44971 every 8 connection(s)
I20260810 18:45:19.281761 27537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
I20260810 18:45:19.282408 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 27537
I20260810 18:45:19.283310 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.195:38771
--local_ip_for_outbound_sockets=127.4.228.195
--tserver_master_addrs=127.4.228.254:45083
--webserver_port=42761
--webserver_interface=127.4.228.195
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:40315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:45:19 dist-test-slave-7kzw krb5kdc[26383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387519, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:45:19.309749 27723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:19.284940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:54485 (local address 127.4.228.254:45083)
0810 18:45:19.285452 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:19.285459 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:19.286960 (+ 1501us) server_negotiation.cc:407] Connection header received
0810 18:45:19.289984 (+ 3024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:19.289989 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:19.290091 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:19.290221 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:19.292755 (+ 2534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:19.293493 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:19.296120 (+ 2627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:19.296879 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:19.297042 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:19.301439 (+ 4397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:19.301448 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:19.301456 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:19.301507 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:19.305691 (+ 4184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:19.306684 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:19.306692 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:19.306699 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:19.306878 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:19.307429 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:19.307437 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:19.307444 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:19.307889 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:19.308051 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:19.308918 (+ 867us) server_negotiation.cc:299] Negotiation successful
0810 18:45:19.309217 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":242,"threads_started":1}
I20260810 18:45:19.310858 27725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:19.285771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.194:54485)
0810 18:45:19.286739 (+ 968us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:19.286804 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:19.289595 (+ 2791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:19.290568 (+ 973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:19.290583 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:19.291584 (+ 1001us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:45:19.292504 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:19.292536 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:19.293679 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:19.293688 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:19.295916 (+ 2228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:19.295939 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:19.297155 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:19.297161 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:19.297376 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:19.298218 (+ 842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:45:19.298253 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:45:19.301144 (+ 2891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:45:19.305889 (+ 4745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:19.305898 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:19.305916 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:19.306446 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:19.307036 (+ 590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:19.307040 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:19.307042 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:19.307290 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:19.308069 (+ 779us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:45:19.308076 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:45:19.308479 (+ 403us) client_negotiation.cc:769] Sending connection context
0810 18:45:19.308774 (+ 295us) client_negotiation.cc:240] Negotiation successful
0810 18:45:19.309243 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":335,"threads_started":1}
I20260810 18:45:19.312614 27720 heartbeater.cc:347] Connected to a master server at 127.4.228.254:45083
I20260810 18:45:19.313045 27720 heartbeater.cc:464] Registering TS with master...
I20260810 18:45:19.314177 27720 heartbeater.cc:511] Master 127.4.228.254:45083 requested a full tablet report, sending...
I20260810 18:45:19.316228 27245 ts_manager.cc:194] Registered new tserver with Master: 108755ecae3544b8b7470da9de6965a6 (127.4.228.194:44971)
I20260810 18:45:19.317104 27245 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.194@KRBTEST.COM'} at 127.4.228.194:54485
W20260810 18:45:19.653157 27727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:45:19.653496 27727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:45:19.653682 27727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:45:19.662775 27727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:45:19.662984 27727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:45:19.663205 27727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.195
I20260810 18:45:19.676115 27727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:40315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.195:38771
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.228.195
--webserver_port=42761
--tserver_master_addrs=127.4.228.254:45083
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.195
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:45:19.677654 27727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:45:19.679464 27727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:45:19.694445 27740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:19.694620 27737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:19.694541 27738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:45:19.695353 27727 server_base.cc:1034] running on GCE node
I20260810 18:45:19.696141 27727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:45:19.697150 27727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:45:19.698375 27727 hybrid_clock.cc:648] HybridClock initialized: now 1786387519698297 us; error 55 us; skew 500 ppm
I20260810 18:45:19.698786 27727 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:45:19 dist-test-slave-7kzw krb5kdc[26383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387519, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:45:19.702586 27727 init.cc:377] Logged in from keytab as kudu/127.4.228.195@KRBTEST.COM (short username kudu)
I20260810 18:45:19.704222 27727 webserver.cc:460] Webserver started at http://127.4.228.195:42761/ using document root <none> and password file <none>
I20260810 18:45:19.704929 27727 fs_manager.cc:359] Metadata directory not provided
I20260810 18:45:19.705047 27727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:45:19.710793 27727 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 18:45:19.713748 27749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:19.715425 27727 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:45:19.715637 27727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/wal
uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd"
format_stamp: "Formatted at 2026-08-10 18:45:16 on dist-test-slave-7kzw"
server_key: "fc452c4f5a67070abd032841211552a2"
server_key_iv: "79d0d0440dd80db90db48c342a8de42b"
server_key_version: "encryptionkey@0"
I20260810 18:45:19.716748 27727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:45:19.742384 27727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:45:19.747063 27727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:45:19.747399 27727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:45:19.748523 27727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:45:19.750038 27727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:45:19.750185 27727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:19.750344 27727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:45:19.750447 27727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:19.776382 27533 heartbeater.cc:503] Master 127.4.228.254:45083 was elected leader, sending a full tablet report...
I20260810 18:45:19.789570 27727 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.195:38771
I20260810 18:45:19.789633 27915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.195:38771 every 8 connection(s)
I20260810 18:45:19.791296 27727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
I20260810 18:45:19.801779 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 27727
Aug 10 18:45:19 dist-test-slave-7kzw krb5kdc[26383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387519, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:45:19.815603 27723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:19.794417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:55909 (local address 127.4.228.254:45083)
0810 18:45:19.794690 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:19.794695 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:19.796020 (+ 1325us) server_negotiation.cc:407] Connection header received
0810 18:45:19.798184 (+ 2164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:19.798190 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:19.798313 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:19.798444 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:19.800191 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:19.800980 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:19.803456 (+ 2476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:19.804249 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:19.804345 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:19.809026 (+ 4681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:19.809035 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:19.809042 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:19.809089 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:19.812057 (+ 2968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:19.813035 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:19.813041 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:19.813047 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:19.813220 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:19.813681 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:19.813686 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:19.813689 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:19.813995 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:19.814202 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:19.814986 (+ 784us) server_negotiation.cc:299] Negotiation successful
0810 18:45:19.815251 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260810 18:45:19.816841 27919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:19.794937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.195:55909)
0810 18:45:19.795837 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:19.795885 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:19.797869 (+ 1984us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:19.798739 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:19.798748 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:19.799318 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:45:19.799946 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:19.799965 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:19.801154 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:19.801163 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:19.803206 (+ 2043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:19.803220 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:19.804770 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:19.804779 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:19.805140 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:19.805955 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:45:19.806003 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:45:19.808750 (+ 2747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:45:19.812206 (+ 3456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:19.812218 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:19.812239 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:19.812893 (+ 654us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:19.813351 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:19.813355 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:19.813357 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:19.813545 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:19.814139 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:45:19.814148 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:45:19.814574 (+ 426us) client_negotiation.cc:769] Sending connection context
0810 18:45:19.814867 (+ 293us) client_negotiation.cc:240] Negotiation successful
0810 18:45:19.815416 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":369,"threads_started":1}
I20260810 18:45:19.818760 27916 heartbeater.cc:347] Connected to a master server at 127.4.228.254:45083
I20260810 18:45:19.819171 27921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:19.803058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50590 (local address 127.4.228.254:45083)
0810 18:45:19.804170 (+ 1112us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:19.804175 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:19.804196 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:45:19.804242 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:19.804245 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:19.804304 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:19.804402 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:19.806722 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:19.807510 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:19.809072 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:19.809568 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:19.809795 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:19.812880 (+ 3085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:19.812884 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:19.812888 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:19.812917 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:19.815949 (+ 3032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:19.816807 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:19.816812 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:19.816814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:19.817035 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:19.817633 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:19.817638 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:19.817643 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:19.817933 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:19.818243 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:19.818586 (+ 343us) server_negotiation.cc:299] Negotiation successful
0810 18:45:19.818822 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1003,"thread_start_us":772,"threads_started":1}
I20260810 18:45:19.819170 27916 heartbeater.cc:464] Registering TS with master...
I20260810 18:45:19.820226 27916 heartbeater.cc:511] Master 127.4.228.254:45083 requested a full tablet report, sending...
I20260810 18:45:19.822117 27245 ts_manager.cc:194] Registered new tserver with Master: a8e4f6f0df3e4d3c8890b77a53cce6dd (127.4.228.195:38771)
I20260810 18:45:19.822968 27245 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.195@KRBTEST.COM'} at 127.4.228.195:55909
I20260810 18:45:19.834758 5011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:45:19.842819 27921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:19.835761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50598 (local address 127.4.228.254:45083)
0810 18:45:19.835947 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:19.835951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:19.836487 (+ 536us) server_negotiation.cc:407] Connection header received
0810 18:45:19.836698 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:19.836702 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:19.836851 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:19.837002 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:45:19.837482 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:19.838271 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:19.840200 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:19.840737 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:19.840844 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:19.841424 (+ 580us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:45:19.842112 (+ 688us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:45:19.842233 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:19.842457 (+ 224us) server_negotiation.cc:299] Negotiation successful
0810 18:45:19.842602 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 18:45:19.843940 27245 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:50598:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:45:19.852169 27245 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:45:19.878597 27931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:19.865682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:38771 (local address 127.0.0.1:40778)
0810 18:45:19.867508 (+ 1826us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:19.867581 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:19.867813 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:19.869966 (+ 2153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:19.869977 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:19.870012 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:45:19.870408 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:19.870419 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:19.873081 (+ 2662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:19.873092 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:19.875856 (+ 2764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:19.875870 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:19.877375 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:19.877395 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:19.877657 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:19.877727 (+ 70us) client_negotiation.cc:769] Sending connection context
0810 18:45:19.877864 (+ 137us) client_negotiation.cc:240] Negotiation successful
0810 18:45:19.878098 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1448,"mutex_wait_us":3,"thread_start_us":6081,"threads_started":1}
I20260810 18:45:19.879992 27933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:19.866601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40778 (local address 127.4.228.195:38771)
0810 18:45:19.869207 (+ 2606us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:19.869228 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:19.869271 (+ 43us) server_negotiation.cc:407] Connection header received
0810 18:45:19.869389 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:19.869400 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:19.869686 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:19.870613 (+ 927us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:45:19.871017 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:19.872818 (+ 1801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:19.876047 (+ 3229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:19.877081 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:19.879187 (+ 2106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:19.879404 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:19.879473 (+ 69us) server_negotiation.cc:299] Negotiation successful
0810 18:45:19.879676 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2401,"thread_start_us":2234,"threads_started":1}
I20260810 18:45:19.881202 27930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:19.867965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:44971 (local address 127.0.0.1:54272)
0810 18:45:19.872123 (+ 4158us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:19.872138 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:19.872292 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:19.873244 (+ 952us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:19.873249 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:19.873274 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:45:19.873814 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:19.873831 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:19.875303 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:19.875311 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:19.877959 (+ 2648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:19.877980 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:19.880326 (+ 2346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:19.880332 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:19.880533 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:19.880573 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 18:45:19.880674 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 18:45:19.880808 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4054,"mutex_wait_us":130,"thread_start_us":3344,"threads_started":1}
I20260810 18:45:19.881305 27938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:19.868262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54272 (local address 127.4.228.194:44971)
0810 18:45:19.871084 (+ 2822us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:19.871096 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:19.872248 (+ 1152us) server_negotiation.cc:407] Connection header received
0810 18:45:19.872474 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:19.872480 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:19.872695 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:19.872903 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:45:19.874049 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:19.875168 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:19.878157 (+ 2989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:19.879783 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:19.879917 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:19.880102 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:19.880778 (+ 676us) server_negotiation.cc:299] Negotiation successful
0810 18:45:19.880924 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2650,"thread_start_us":2488,"threads_started":1}
I20260810 18:45:19.882377 27935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:19.865687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:42335 (local address 127.0.0.1:41006)
0810 18:45:19.871627 (+ 5940us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:19.871652 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:19.871911 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:19.872885 (+ 974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:19.872892 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:19.872925 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:45:19.873434 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:19.873449 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:19.875637 (+ 2188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:19.875643 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:19.878009 (+ 2366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:19.878022 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:19.881532 (+ 3510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:19.881539 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:19.881740 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:19.881773 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 18:45:19.881873 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 18:45:19.881996 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5769,"mutex_wait_us":309,"thread_start_us":1005,"threads_started":1}
I20260810 18:45:19.882583 27932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:19.866119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41006 (local address 127.4.228.193:42335)
0810 18:45:19.868596 (+ 2477us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:19.868604 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:19.871769 (+ 3165us) server_negotiation.cc:407] Connection header received
0810 18:45:19.872137 (+ 368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:19.872146 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:19.872385 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:19.872572 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:45:19.873736 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:19.875468 (+ 1732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:19.878183 (+ 2715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:19.879782 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:19.879919 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:19.880138 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:19.881971 (+ 1833us) server_negotiation.cc:299] Negotiation successful
0810 18:45:19.882168 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2306,"thread_start_us":2114,"threads_started":1}
I20260810 18:45:19.884696 27625 tablet_service.cc:1467] Processing CreateTablet for tablet 68525487e52345708d37cb880fb88a5d (DEFAULT_TABLE table=test-table [id=7a388ea8313d463bb2c9af66242f96be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:45:19.884716 27817 tablet_service.cc:1467] Processing CreateTablet for tablet 68525487e52345708d37cb880fb88a5d (DEFAULT_TABLE table=test-table [id=7a388ea8313d463bb2c9af66242f96be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:45:19.885890 27625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68525487e52345708d37cb880fb88a5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:45:19.886296 27817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68525487e52345708d37cb880fb88a5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:45:19.886535 27432 tablet_service.cc:1467] Processing CreateTablet for tablet 68525487e52345708d37cb880fb88a5d (DEFAULT_TABLE table=test-table [id=7a388ea8313d463bb2c9af66242f96be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:45:19.888103 27432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68525487e52345708d37cb880fb88a5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:45:19.900204 27939 tablet_bootstrap.cc:492] T 68525487e52345708d37cb880fb88a5d P 108755ecae3544b8b7470da9de6965a6: Bootstrap starting.
I20260810 18:45:19.905014 27939 tablet_bootstrap.cc:654] T 68525487e52345708d37cb880fb88a5d P 108755ecae3544b8b7470da9de6965a6: Neither blocks nor log segments found. Creating new log.
I20260810 18:45:19.905591 27941 tablet_bootstrap.cc:492] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f: Bootstrap starting.
I20260810 18:45:19.907449 27939 log.cc:824] T 68525487e52345708d37cb880fb88a5d P 108755ecae3544b8b7470da9de6965a6: Log is configured to *not* fsync() on all Append() calls
I20260810 18:45:19.910485 27941 tablet_bootstrap.cc:654] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f: Neither blocks nor log segments found. Creating new log.
I20260810 18:45:19.910807 27940 tablet_bootstrap.cc:492] T 68525487e52345708d37cb880fb88a5d P a8e4f6f0df3e4d3c8890b77a53cce6dd: Bootstrap starting.
I20260810 18:45:19.911720 27939 tablet_bootstrap.cc:492] T 68525487e52345708d37cb880fb88a5d P 108755ecae3544b8b7470da9de6965a6: No bootstrap required, opened a new log
I20260810 18:45:19.912504 27941 log.cc:824] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f: Log is configured to *not* fsync() on all Append() calls
I20260810 18:45:19.912883 27939 ts_tablet_manager.cc:1397] T 68525487e52345708d37cb880fb88a5d P 108755ecae3544b8b7470da9de6965a6: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.000s
I20260810 18:45:19.914716 27940 tablet_bootstrap.cc:654] T 68525487e52345708d37cb880fb88a5d P a8e4f6f0df3e4d3c8890b77a53cce6dd: Neither blocks nor log segments found. Creating new log.
I20260810 18:45:19.915820 27940 log.cc:824] T 68525487e52345708d37cb880fb88a5d P a8e4f6f0df3e4d3c8890b77a53cce6dd: Log is configured to *not* fsync() on all Append() calls
I20260810 18:45:19.916954 27941 tablet_bootstrap.cc:492] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f: No bootstrap required, opened a new log
I20260810 18:45:19.917320 27941 ts_tablet_manager.cc:1397] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260810 18:45:19.921408 27940 tablet_bootstrap.cc:492] T 68525487e52345708d37cb880fb88a5d P a8e4f6f0df3e4d3c8890b77a53cce6dd: No bootstrap required, opened a new log
I20260810 18:45:19.921712 27940 ts_tablet_manager.cc:1397] T 68525487e52345708d37cb880fb88a5d P a8e4f6f0df3e4d3c8890b77a53cce6dd: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20260810 18:45:19.924965 27939 raft_consensus.cc:348] T 68525487e52345708d37cb880fb88a5d P 108755ecae3544b8b7470da9de6965a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6114ee077fd140739ff7a63019042a8f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42335 } } peers { permanent_uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38771 } } peers { permanent_uuid: "108755ecae3544b8b7470da9de6965a6" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44971 } }
I20260810 18:45:19.925539 27939 raft_consensus.cc:374] T 68525487e52345708d37cb880fb88a5d P 108755ecae3544b8b7470da9de6965a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:45:19.925670 27939 raft_consensus.cc:729] T 68525487e52345708d37cb880fb88a5d P 108755ecae3544b8b7470da9de6965a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 108755ecae3544b8b7470da9de6965a6, State: Initialized, Role: FOLLOWER
I20260810 18:45:19.926440 27939 consensus_queue.cc:260] T 68525487e52345708d37cb880fb88a5d P 108755ecae3544b8b7470da9de6965a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6114ee077fd140739ff7a63019042a8f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42335 } } peers { permanent_uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38771 } } peers { permanent_uuid: "108755ecae3544b8b7470da9de6965a6" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44971 } }
I20260810 18:45:19.927910 27720 heartbeater.cc:503] Master 127.4.228.254:45083 was elected leader, sending a full tablet report...
I20260810 18:45:19.928771 27939 ts_tablet_manager.cc:1428] T 68525487e52345708d37cb880fb88a5d P 108755ecae3544b8b7470da9de6965a6: Time spent starting tablet: real 0.016s user 0.008s sys 0.007s
I20260810 18:45:19.929131 27941 raft_consensus.cc:348] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6114ee077fd140739ff7a63019042a8f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42335 } } peers { permanent_uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38771 } } peers { permanent_uuid: "108755ecae3544b8b7470da9de6965a6" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44971 } }
I20260810 18:45:19.929726 27941 raft_consensus.cc:374] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:45:19.929872 27941 raft_consensus.cc:729] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6114ee077fd140739ff7a63019042a8f, State: Initialized, Role: FOLLOWER
I20260810 18:45:19.930645 27941 consensus_queue.cc:260] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6114ee077fd140739ff7a63019042a8f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42335 } } peers { permanent_uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38771 } } peers { permanent_uuid: "108755ecae3544b8b7470da9de6965a6" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44971 } }
I20260810 18:45:19.931391 27940 raft_consensus.cc:348] T 68525487e52345708d37cb880fb88a5d P a8e4f6f0df3e4d3c8890b77a53cce6dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6114ee077fd140739ff7a63019042a8f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42335 } } peers { permanent_uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38771 } } peers { permanent_uuid: "108755ecae3544b8b7470da9de6965a6" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44971 } }
I20260810 18:45:19.931839 27940 raft_consensus.cc:374] T 68525487e52345708d37cb880fb88a5d P a8e4f6f0df3e4d3c8890b77a53cce6dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:45:19.931946 27940 raft_consensus.cc:729] T 68525487e52345708d37cb880fb88a5d P a8e4f6f0df3e4d3c8890b77a53cce6dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8e4f6f0df3e4d3c8890b77a53cce6dd, State: Initialized, Role: FOLLOWER
I20260810 18:45:19.932612 27941 ts_tablet_manager.cc:1428] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
I20260810 18:45:19.932550 27940 consensus_queue.cc:260] T 68525487e52345708d37cb880fb88a5d P a8e4f6f0df3e4d3c8890b77a53cce6dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6114ee077fd140739ff7a63019042a8f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42335 } } peers { permanent_uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38771 } } peers { permanent_uuid: "108755ecae3544b8b7470da9de6965a6" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44971 } }
I20260810 18:45:19.933847 27916 heartbeater.cc:503] Master 127.4.228.254:45083 was elected leader, sending a full tablet report...
I20260810 18:45:19.934448 27940 ts_tablet_manager.cc:1428] T 68525487e52345708d37cb880fb88a5d P a8e4f6f0df3e4d3c8890b77a53cce6dd: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
W20260810 18:45:19.985674 27534 tablet.cc:2369] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:45:20.012138 27950 raft_consensus.cc:482] T 68525487e52345708d37cb880fb88a5d P a8e4f6f0df3e4d3c8890b77a53cce6dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:45:20.012539 27950 raft_consensus.cc:504] T 68525487e52345708d37cb880fb88a5d P a8e4f6f0df3e4d3c8890b77a53cce6dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6114ee077fd140739ff7a63019042a8f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42335 } } peers { permanent_uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38771 } } peers { permanent_uuid: "108755ecae3544b8b7470da9de6965a6" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44971 } }
I20260810 18:45:20.014425 27950 leader_election.cc:288] T 68525487e52345708d37cb880fb88a5d P a8e4f6f0df3e4d3c8890b77a53cce6dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6114ee077fd140739ff7a63019042a8f (127.4.228.193:42335), 108755ecae3544b8b7470da9de6965a6 (127.4.228.194:44971)
I20260810 18:45:20.021203 27919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:20.014718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:42335 (local address 127.4.228.195:37975)
0810 18:45:20.014927 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:20.014948 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:20.015087 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:20.015611 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:20.015623 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:20.015653 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:45:20.015899 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:20.015913 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:20.017294 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:20.017300 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:20.018943 (+ 1643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:20.018956 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:20.020338 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:20.020344 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:20.020557 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:20.020622 (+ 65us) client_negotiation.cc:769] Sending connection context
0810 18:45:20.020740 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 18:45:20.020883 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 18:45:20.021224 27932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:20.014888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:37975 (local address 127.4.228.193:42335)
0810 18:45:20.015127 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:20.015133 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:20.015157 (+ 24us) server_negotiation.cc:407] Connection header received
0810 18:45:20.015275 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:20.015280 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:20.015367 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:20.015486 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:45:20.016090 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:20.017146 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:20.019121 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:20.020149 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:20.020313 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:20.020384 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:20.020805 (+ 421us) server_negotiation.cc:299] Negotiation successful
0810 18:45:20.020949 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 18:45:20.021770 27938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:20.015280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:48367 (local address 127.4.228.194:44971)
0810 18:45:20.015504 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:20.015509 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:20.015923 (+ 414us) server_negotiation.cc:407] Connection header received
0810 18:45:20.016013 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:20.016017 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:20.016101 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:20.016171 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:45:20.016837 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:20.017760 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:20.019706 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:20.020692 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:20.020842 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:20.020909 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:20.021350 (+ 441us) server_negotiation.cc:299] Negotiation successful
0810 18:45:20.021485 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 18:45:20.021821 27952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:20.015104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:44971 (local address 127.4.228.195:48367)
0810 18:45:20.015795 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:20.015810 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:20.015913 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:20.016224 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:20.016230 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:20.016256 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:45:20.016645 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:20.016658 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:20.017932 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:20.017935 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:20.019525 (+ 1590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:20.019541 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:20.020922 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:20.020929 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:20.021147 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:20.021181 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 18:45:20.021290 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 18:45:20.021475 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":388,"threads_started":1}
I20260810 18:45:20.022666 27465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68525487e52345708d37cb880fb88a5d" candidate_uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6114ee077fd140739ff7a63019042a8f" is_pre_election: true
I20260810 18:45:20.023049 27656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68525487e52345708d37cb880fb88a5d" candidate_uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "108755ecae3544b8b7470da9de6965a6" is_pre_election: true
I20260810 18:45:20.023373 27465 raft_consensus.cc:2454] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8e4f6f0df3e4d3c8890b77a53cce6dd in term 0.
I20260810 18:45:20.023638 27656 raft_consensus.cc:2454] T 68525487e52345708d37cb880fb88a5d P 108755ecae3544b8b7470da9de6965a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8e4f6f0df3e4d3c8890b77a53cce6dd in term 0.
I20260810 18:45:20.024235 27750 leader_election.cc:302] T 68525487e52345708d37cb880fb88a5d P a8e4f6f0df3e4d3c8890b77a53cce6dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6114ee077fd140739ff7a63019042a8f, a8e4f6f0df3e4d3c8890b77a53cce6dd; no voters:
I20260810 18:45:20.024726 27950 raft_consensus.cc:2781] T 68525487e52345708d37cb880fb88a5d P a8e4f6f0df3e4d3c8890b77a53cce6dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:45:20.024914 27950 raft_consensus.cc:482] T 68525487e52345708d37cb880fb88a5d P a8e4f6f0df3e4d3c8890b77a53cce6dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:45:20.025033 27950 raft_consensus.cc:3037] T 68525487e52345708d37cb880fb88a5d P a8e4f6f0df3e4d3c8890b77a53cce6dd [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:45:20.027678 27950 raft_consensus.cc:504] T 68525487e52345708d37cb880fb88a5d P a8e4f6f0df3e4d3c8890b77a53cce6dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6114ee077fd140739ff7a63019042a8f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42335 } } peers { permanent_uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38771 } } peers { permanent_uuid: "108755ecae3544b8b7470da9de6965a6" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44971 } }
I20260810 18:45:20.028502 27950 leader_election.cc:288] T 68525487e52345708d37cb880fb88a5d P a8e4f6f0df3e4d3c8890b77a53cce6dd [CANDIDATE]: Term 1 election: Requested vote from peers 6114ee077fd140739ff7a63019042a8f (127.4.228.193:42335), 108755ecae3544b8b7470da9de6965a6 (127.4.228.194:44971)
I20260810 18:45:20.029127 27465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68525487e52345708d37cb880fb88a5d" candidate_uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6114ee077fd140739ff7a63019042a8f"
I20260810 18:45:20.029176 27656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68525487e52345708d37cb880fb88a5d" candidate_uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "108755ecae3544b8b7470da9de6965a6"
I20260810 18:45:20.029377 27465 raft_consensus.cc:3037] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:45:20.029421 27656 raft_consensus.cc:3037] T 68525487e52345708d37cb880fb88a5d P 108755ecae3544b8b7470da9de6965a6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:45:20.032034 27465 raft_consensus.cc:2454] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8e4f6f0df3e4d3c8890b77a53cce6dd in term 1.
I20260810 18:45:20.032032 27656 raft_consensus.cc:2454] T 68525487e52345708d37cb880fb88a5d P 108755ecae3544b8b7470da9de6965a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8e4f6f0df3e4d3c8890b77a53cce6dd in term 1.
I20260810 18:45:20.032559 27750 leader_election.cc:302] T 68525487e52345708d37cb880fb88a5d P a8e4f6f0df3e4d3c8890b77a53cce6dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6114ee077fd140739ff7a63019042a8f, a8e4f6f0df3e4d3c8890b77a53cce6dd; no voters:
I20260810 18:45:20.032919 27950 raft_consensus.cc:2781] T 68525487e52345708d37cb880fb88a5d P a8e4f6f0df3e4d3c8890b77a53cce6dd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:45:20.033448 27950 raft_consensus.cc:686] T 68525487e52345708d37cb880fb88a5d P a8e4f6f0df3e4d3c8890b77a53cce6dd [term 1 LEADER]: Becoming Leader. State: Replica: a8e4f6f0df3e4d3c8890b77a53cce6dd, State: Running, Role: LEADER
I20260810 18:45:20.034067 27950 consensus_queue.cc:237] T 68525487e52345708d37cb880fb88a5d P a8e4f6f0df3e4d3c8890b77a53cce6dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6114ee077fd140739ff7a63019042a8f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42335 } } peers { permanent_uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38771 } } peers { permanent_uuid: "108755ecae3544b8b7470da9de6965a6" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44971 } }
W20260810 18:45:20.034330 27721 tablet.cc:2369] T 68525487e52345708d37cb880fb88a5d P 108755ecae3544b8b7470da9de6965a6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:45:20.039389 27245 catalog_manager.cc:5495] T 68525487e52345708d37cb880fb88a5d P a8e4f6f0df3e4d3c8890b77a53cce6dd reported cstate change: term changed from 0 to 1, leader changed from <none> to a8e4f6f0df3e4d3c8890b77a53cce6dd (127.4.228.195). New cstate: current_term: 1 leader_uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6114ee077fd140739ff7a63019042a8f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "108755ecae3544b8b7470da9de6965a6" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44971 } health_report { overall_health: UNKNOWN } } }
W20260810 18:45:20.043272 27918 tablet.cc:2369] T 68525487e52345708d37cb880fb88a5d P a8e4f6f0df3e4d3c8890b77a53cce6dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:45:20.079106 27933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:20.073336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40792 (local address 127.4.228.195:38771)
0810 18:45:20.073548 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:20.073553 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:20.073572 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:45:20.073750 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:20.073756 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:20.073853 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:20.073983 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:45:20.074377 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:20.075188 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:20.076755 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:20.077145 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:20.077246 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:20.077775 (+ 529us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:45:20.078310 (+ 535us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:45:20.078429 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:20.078736 (+ 307us) server_negotiation.cc:299] Negotiation successful
0810 18:45:20.078844 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 18:45:20.087812 27465 raft_consensus.cc:1261] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [term 1 FOLLOWER]: Refusing update from remote peer a8e4f6f0df3e4d3c8890b77a53cce6dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:45:20.087811 27656 raft_consensus.cc:1261] T 68525487e52345708d37cb880fb88a5d P 108755ecae3544b8b7470da9de6965a6 [term 1 FOLLOWER]: Refusing update from remote peer a8e4f6f0df3e4d3c8890b77a53cce6dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:45:20.088853 27953 consensus_queue.cc:1035] T 68525487e52345708d37cb880fb88a5d P a8e4f6f0df3e4d3c8890b77a53cce6dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6114ee077fd140739ff7a63019042a8f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:45:20.089274 27950 consensus_queue.cc:1035] T 68525487e52345708d37cb880fb88a5d P a8e4f6f0df3e4d3c8890b77a53cce6dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "108755ecae3544b8b7470da9de6965a6" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:45:20.117990 27243 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:50598:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:45:20.118381 27243 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:50598:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:45:20.122740 27243 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198: Sending DeleteTablet for 3 replicas of tablet 68525487e52345708d37cb880fb88a5d
I20260810 18:45:20.124147 27817 tablet_service.cc:1514] Processing DeleteTablet for tablet 68525487e52345708d37cb880fb88a5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:45:20 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:40778
I20260810 18:45:20.124178 27432 tablet_service.cc:1514] Processing DeleteTablet for tablet 68525487e52345708d37cb880fb88a5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:45:20 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:41006
I20260810 18:45:20.124426 27625 tablet_service.cc:1514] Processing DeleteTablet for tablet 68525487e52345708d37cb880fb88a5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:45:20 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:54272
I20260810 18:45:20.125408 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 27302
I20260810 18:45:20.137913 27973 tablet_replica.cc:330] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f: stopping tablet replica
I20260810 18:45:20.138713 27973 raft_consensus.cc:2229] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:45:20.139609 27973 raft_consensus.cc:2258] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:45:20.140775 27971 tablet_replica.cc:330] T 68525487e52345708d37cb880fb88a5d P a8e4f6f0df3e4d3c8890b77a53cce6dd: stopping tablet replica
I20260810 18:45:20.141052 27974 tablet_replica.cc:330] T 68525487e52345708d37cb880fb88a5d P 108755ecae3544b8b7470da9de6965a6: stopping tablet replica
I20260810 18:45:20.141281 27971 raft_consensus.cc:2229] T 68525487e52345708d37cb880fb88a5d P a8e4f6f0df3e4d3c8890b77a53cce6dd [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:45:20.141485 27974 raft_consensus.cc:2229] T 68525487e52345708d37cb880fb88a5d P 108755ecae3544b8b7470da9de6965a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:45:20.142174 27971 raft_consensus.cc:2258] T 68525487e52345708d37cb880fb88a5d P a8e4f6f0df3e4d3c8890b77a53cce6dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:45:20.142230 27974 raft_consensus.cc:2258] T 68525487e52345708d37cb880fb88a5d P 108755ecae3544b8b7470da9de6965a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:45:20.144662 27974 ts_tablet_manager.cc:1905] T 68525487e52345708d37cb880fb88a5d P 108755ecae3544b8b7470da9de6965a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:45:20.144673 27971 ts_tablet_manager.cc:1905] T 68525487e52345708d37cb880fb88a5d P a8e4f6f0df3e4d3c8890b77a53cce6dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:45:20.152226 27974 ts_tablet_manager.cc:1918] T 68525487e52345708d37cb880fb88a5d P 108755ecae3544b8b7470da9de6965a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:45:20.152419 27974 log.cc:1196] T 68525487e52345708d37cb880fb88a5d P 108755ecae3544b8b7470da9de6965a6: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/wal/wals/68525487e52345708d37cb880fb88a5d
I20260810 18:45:20.152748 27971 ts_tablet_manager.cc:1918] T 68525487e52345708d37cb880fb88a5d P a8e4f6f0df3e4d3c8890b77a53cce6dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:45:20.152951 27971 log.cc:1196] T 68525487e52345708d37cb880fb88a5d P a8e4f6f0df3e4d3c8890b77a53cce6dd: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/wal/wals/68525487e52345708d37cb880fb88a5d
I20260810 18:45:20.153049 27974 ts_tablet_manager.cc:1939] T 68525487e52345708d37cb880fb88a5d P 108755ecae3544b8b7470da9de6965a6: Deleting consensus metadata
I20260810 18:45:20.153612 27971 ts_tablet_manager.cc:1939] T 68525487e52345708d37cb880fb88a5d P a8e4f6f0df3e4d3c8890b77a53cce6dd: Deleting consensus metadata
I20260810 18:45:20.155170 27222 catalog_manager.cc:4841] TS 108755ecae3544b8b7470da9de6965a6 (127.4.228.194:44971): tablet 68525487e52345708d37cb880fb88a5d (table test-table [id=7a388ea8313d463bb2c9af66242f96be]) successfully deleted
I20260810 18:45:20.155503 27223 catalog_manager.cc:4841] TS a8e4f6f0df3e4d3c8890b77a53cce6dd (127.4.228.195:38771): tablet 68525487e52345708d37cb880fb88a5d (table test-table [id=7a388ea8313d463bb2c9af66242f96be]) successfully deleted
I20260810 18:45:20.156342 27932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:20.126607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41016 (local address 127.4.228.193:42335)
0810 18:45:20.127414 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:20.127424 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:20.127448 (+ 24us) server_negotiation.cc:407] Connection header received
0810 18:45:20.127522 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:20.127530 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:20.127632 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:20.127751 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:20.130318 (+ 2567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:20.131905 (+ 1587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:20.134582 (+ 2677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:20.135664 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:20.135832 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:20.146275 (+ 10443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:20.146292 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:20.146312 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:20.146372 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:20.150691 (+ 4319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:20.151729 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:20.151740 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:20.151745 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:20.151919 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:20.152611 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:20.152616 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:20.152618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:20.153437 (+ 819us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:20.155138 (+ 1701us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:20.155884 (+ 746us) server_negotiation.cc:299] Negotiation successful
0810 18:45:20.156074 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620}
I20260810 18:45:20.157119 27528 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:45:20.157157 27973 ts_tablet_manager.cc:1905] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:45:20.251739 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 27302
W20260810 18:45:20.279850 27222 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.228.193:42335 (error 108)
I20260810 18:45:20.280881 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 27537
I20260810 18:45:20.284034 27935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:20.283477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:42335 (local address 127.0.0.1:41022)
0810 18:45:20.283739 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:20.283862 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.193:42335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
W20260810 18:45:20.284477 27222 catalog_manager.cc:4584] TS 6114ee077fd140739ff7a63019042a8f (127.4.228.193:42335): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 68525487e52345708d37cb880fb88a5d: Network error: Client connection negotiation failed: client connection to 127.4.228.193:42335: connect: Connection refused (error 111)
I20260810 18:45:20.299870 27938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:20.281714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54276 (local address 127.4.228.194:44971)
0810 18:45:20.281918 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:20.281925 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:20.282085 (+ 160us) server_negotiation.cc:407] Connection header received
0810 18:45:20.282292 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:20.282298 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:20.282387 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:20.282561 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:20.284378 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:20.285593 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:20.287720 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:20.288360 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:20.288508 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:20.292149 (+ 3641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:20.292162 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:20.292182 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:20.292224 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:20.296295 (+ 4071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:20.297160 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:20.297171 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:20.297176 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:20.297320 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:20.297751 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:20.297757 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:20.297759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:20.298291 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:20.298623 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:20.299089 (+ 466us) server_negotiation.cc:299] Negotiation successful
0810 18:45:20.299312 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 18:45:20.300567 27713 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:45:20.338351 27935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:20.337712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:42335 (local address 127.0.0.1:41036)
0810 18:45:20.337992 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:20.338126 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.193:42335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260810 18:45:20.339608 27935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:20.339183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:42335 (local address 127.0.0.1:41042)
0810 18:45:20.339360 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:20.339468 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.193:42335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260810 18:45:20.391379 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 27537
I20260810 18:45:20.400379 27935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:20.399884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:42335 (local address 127.0.0.1:41044)
0810 18:45:20.400120 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:20.400211 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.193:42335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260810 18:45:20.401576 27935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:20.401203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:42335 (local address 127.0.0.1:41058)
0810 18:45:20.401369 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:20.401451 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.193:42335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 18:45:20.430553 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 27727
I20260810 18:45:20.449371 27933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:20.431729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40808 (local address 127.4.228.195:38771)
0810 18:45:20.431960 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:20.431966 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:20.431992 (+ 26us) server_negotiation.cc:407] Connection header received
0810 18:45:20.432077 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:20.432085 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:20.432183 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:20.432328 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:20.433840 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:20.435157 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:20.436641 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:20.437353 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:20.437503 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:20.440717 (+ 3214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:20.440735 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:20.440760 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:20.440819 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:20.445429 (+ 4610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:20.446377 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:20.446386 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:20.446391 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:20.446803 (+ 412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:20.447354 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:20.447360 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:20.447363 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:20.447878 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:20.448221 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:20.448608 (+ 387us) server_negotiation.cc:299] Negotiation successful
0810 18:45:20.448920 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 18:45:20.450446 27910 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:45:20.482827 27935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:20.482182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:42335 (local address 127.0.0.1:41074)
0810 18:45:20.482440 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:20.482595 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.193:42335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260810 18:45:20.484069 27935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:20.483605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:42335 (local address 127.0.0.1:41078)
0810 18:45:20.483833 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:20.483924 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.193:42335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260810 18:45:20.575407 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 27727
I20260810 18:45:20.609416 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 27195
I20260810 18:45:20.610801 27293 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:45:20.735651 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 27195
I20260810 18:45:20.768748 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:45083
--webserver_interface=127.4.228.254
--webserver_port=33247
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:40315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:45083
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:45:21.143023 27981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:45:21.143364 27981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:45:21.143469 27981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:45:21.151542 27981 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:45:21.151676 27981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:45:21.151767 27981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:45:21.151826 27981 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:45:21.151875 27981 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:45:21.161971 27981 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:40315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:45083
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:45083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=33247
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:45:21.163422 27981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:45:21.164776 27981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:45:21.177182 27991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:21.177345 27988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:21.177129 27989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:45:21.178530 27981 server_base.cc:1034] running on GCE node
I20260810 18:45:21.179536 27981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:45:21.181907 27981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:45:21.183176 27981 hybrid_clock.cc:648] HybridClock initialized: now 1786387521183103 us; error 60 us; skew 500 ppm
I20260810 18:45:21.183557 27981 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:45:21 dist-test-slave-7kzw krb5kdc[26383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387521, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:45:21.186909 27981 init.cc:377] Logged in from keytab as kudu/127.4.228.254@KRBTEST.COM (short username kudu)
I20260810 18:45:21.188549 27981 webserver.cc:460] Webserver started at http://127.4.228.254:33247/ using document root <none> and password file <none>
I20260810 18:45:21.189388 27981 fs_manager.cc:359] Metadata directory not provided
I20260810 18:45:21.189545 27981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:45:21.196281 27981 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 18:45:21.199694 28003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:21.201311 27981 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:45:21.201529 27981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "346a38b450e0436191ddbba9e03b1198"
format_stamp: "Formatted at 2026-08-10 18:45:14 on dist-test-slave-7kzw"
server_key: "0776f4b220826e5549b8e15679d14529"
server_key_iv: "f27d06aa6bfe6f501895bb753c210983"
server_key_version: "encryptionkey@0"
I20260810 18:45:21.202679 27981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:45:21.222855 27981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:45:21.227377 27981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:45:21.227669 27981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:45:21.248278 27981 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:45083
I20260810 18:45:21.248322 28086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:45083 every 8 connection(s)
I20260810 18:45:21.249922 27981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:45:21.250736 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 27981
I20260810 18:45:21.251935 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.193:42335
--local_ip_for_outbound_sockets=127.4.228.193
--tserver_master_addrs=127.4.228.254:45083
--webserver_port=41535
--webserver_interface=127.4.228.193
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:40315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:45:21.257936 28087 sys_catalog.cc:263] Verifying existing consensus state
I20260810 18:45:21.261286 28087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198: Bootstrap starting.
I20260810 18:45:21.281994 28087 log.cc:824] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198: Log is configured to *not* fsync() on all Append() calls
I20260810 18:45:21.300933 28087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:45:21.301546 28087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198: Bootstrap complete.
I20260810 18:45:21.311508 28087 raft_consensus.cc:348] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "346a38b450e0436191ddbba9e03b1198" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45083 } }
I20260810 18:45:21.312434 28087 raft_consensus.cc:729] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 346a38b450e0436191ddbba9e03b1198, State: Initialized, Role: FOLLOWER
I20260810 18:45:21.313129 28087 consensus_queue.cc:260] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "346a38b450e0436191ddbba9e03b1198" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45083 } }
I20260810 18:45:21.313375 28087 raft_consensus.cc:388] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:45:21.313505 28087 raft_consensus.cc:482] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:45:21.313715 28087 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [term 2 FOLLOWER]: Advancing to term 3
I20260810 18:45:21.317826 28087 raft_consensus.cc:504] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "346a38b450e0436191ddbba9e03b1198" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45083 } }
I20260810 18:45:21.318363 28087 leader_election.cc:302] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 346a38b450e0436191ddbba9e03b1198; no voters:
I20260810 18:45:21.319106 28087 leader_election.cc:288] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 18:45:21.319265 28094 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 18:45:21.320384 28094 raft_consensus.cc:686] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [term 3 LEADER]: Becoming Leader. State: Replica: 346a38b450e0436191ddbba9e03b1198, State: Running, Role: LEADER
I20260810 18:45:21.320979 28094 consensus_queue.cc:237] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "346a38b450e0436191ddbba9e03b1198" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45083 } }
I20260810 18:45:21.321434 28087 sys_catalog.cc:564] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:45:21.323729 28096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 346a38b450e0436191ddbba9e03b1198. Latest consensus state: current_term: 3 leader_uuid: "346a38b450e0436191ddbba9e03b1198" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "346a38b450e0436191ddbba9e03b1198" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45083 } } }
I20260810 18:45:21.324159 28096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [sys.catalog]: This master's current role is: LEADER
I20260810 18:45:21.324638 28095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "346a38b450e0436191ddbba9e03b1198" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "346a38b450e0436191ddbba9e03b1198" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 45083 } } }
I20260810 18:45:21.325069 28095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 346a38b450e0436191ddbba9e03b1198 [sys.catalog]: This master's current role is: LEADER
I20260810 18:45:21.328141 28108 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:45:21.337587 28108 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:45:21.339120 28108 catalog_manager.cc:1259] Loaded cluster ID: 17650ec17c2247408e130a2363f61f0b
I20260810 18:45:21.339238 28108 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260810 18:45:21.341379 28108 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f8980ba59ad google::LogMessage::Fail() at ??:0
@ 0x7f8980ba888a google::LogMessage::SendToLog() at ??:0
@ 0x7f8980ba54de google::LogMessage::Flush() at ??:0
I20260810 18:45:21.348830 28121 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f8980ba6a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f89965d8512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f89965d5976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f899665d610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f899665d439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f89837f1345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f89837f491c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f89837f4799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f89837cfeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f8986aa96db start_thread at ??:0
@ 0x7f897fa3f61f clone at ??:0
W20260810 18:45:21.675868 28089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:45:21.676160 28089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:45:21.676321 28089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:45:21.684504 28089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:45:21.684614 28089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:45:21.684809 28089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.193
I20260810 18:45:21.694823 28089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:40315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.193:42335
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.228.193
--webserver_port=41535
--tserver_master_addrs=127.4.228.254:45083
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.193
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:45:21.696295 28089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:45:21.697746 28089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:45:21.711390 28156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:21.711493 28158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:21.711442 28163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:45:21.712728 28089 server_base.cc:1034] running on GCE node
I20260810 18:45:21.713438 28089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:45:21.714599 28089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:45:21.715874 28089 hybrid_clock.cc:648] HybridClock initialized: now 1786387521715820 us; error 40 us; skew 500 ppm
I20260810 18:45:21.716226 28089 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:45:21 dist-test-slave-7kzw krb5kdc[26383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387521, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:45:21.719681 28089 init.cc:377] Logged in from keytab as kudu/127.4.228.193@KRBTEST.COM (short username kudu)
I20260810 18:45:21.721547 28089 webserver.cc:460] Webserver started at http://127.4.228.193:41535/ using document root <none> and password file <none>
I20260810 18:45:21.722533 28089 fs_manager.cc:359] Metadata directory not provided
I20260810 18:45:21.722718 28089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:45:21.728380 28089 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:45:21.731995 28172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:21.733474 28089 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:45:21.733708 28089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/wal
uuid: "6114ee077fd140739ff7a63019042a8f"
format_stamp: "Formatted at 2026-08-10 18:45:15 on dist-test-slave-7kzw"
server_key: "030db39962bc74b50adf6bed4baf5176"
server_key_iv: "e4870105dfe0384426f9f00d9ed082d9"
server_key_version: "encryptionkey@0"
I20260810 18:45:21.734936 28089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:45:21.760793 28089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:45:21.765256 28089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:45:21.765576 28089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:45:21.766770 28089 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:45:21.768947 28182 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260810 18:45:21.775470 28089 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 18:45:21.775640 28089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260810 18:45:21.775822 28089 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 18:45:21.779191 28089 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 18:45:21.779322 28089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.003s sys 0.000s
I20260810 18:45:21.779668 28182 tablet_bootstrap.cc:492] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f: Bootstrap starting.
I20260810 18:45:21.789222 28182 log.cc:824] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f: Log is configured to *not* fsync() on all Append() calls
I20260810 18:45:21.802971 28182 tablet_bootstrap.cc:492] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:45:21.803891 28182 tablet_bootstrap.cc:492] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f: Bootstrap complete.
I20260810 18:45:21.804497 28182 ts_tablet_manager.cc:1397] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f: Time spent bootstrapping tablet: real 0.025s user 0.017s sys 0.006s
I20260810 18:45:21.815894 28182 raft_consensus.cc:348] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6114ee077fd140739ff7a63019042a8f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42335 } } peers { permanent_uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38771 } } peers { permanent_uuid: "108755ecae3544b8b7470da9de6965a6" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44971 } }
I20260810 18:45:21.817287 28182 raft_consensus.cc:729] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6114ee077fd140739ff7a63019042a8f, State: Initialized, Role: FOLLOWER
I20260810 18:45:21.818343 28182 consensus_queue.cc:260] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6114ee077fd140739ff7a63019042a8f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42335 } } peers { permanent_uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38771 } } peers { permanent_uuid: "108755ecae3544b8b7470da9de6965a6" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44971 } }
I20260810 18:45:21.820369 28182 ts_tablet_manager.cc:1428] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f: Time spent starting tablet: real 0.016s user 0.008s sys 0.007s
I20260810 18:45:21.828691 28089 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.193:42335
I20260810 18:45:21.828729 28346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.193:42335 every 8 connection(s)
I20260810 18:45:21.830045 28089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
I20260810 18:45:21.834919 28349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:21.832923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.193:60895)
0810 18:45:21.833712 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:21.833968 (+ 256us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":332,"threads_started":1}
W20260810 18:45:21.835623 28347 heartbeater.cc:644] Failed to heartbeat to 127.4.228.254:45083 (0 consecutive failures): Network error: Failed to ping master at 127.4.228.254:45083: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
I20260810 18:45:21.836474 28349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:21.836118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.193:44881)
0810 18:45:21.836290 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:21.836357 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 18:45:21.836460 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 28089
I20260810 18:45:21.837181 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.194:44971
--local_ip_for_outbound_sockets=127.4.228.194
--tserver_master_addrs=127.4.228.254:45083
--webserver_port=44003
--webserver_interface=127.4.228.194
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:40315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:45:21.837491 28349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:21.837062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.193:32817)
0810 18:45:21.837213 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:21.837346 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 18:45:21.837739 28347 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 18:45:22.199581 28350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:45:22.199892 28350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:45:22.200052 28350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:45:22.207996 28350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:45:22.208120 28350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:45:22.208319 28350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.194
I20260810 18:45:22.218698 28350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:40315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.194:44971
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.228.194
--webserver_port=44003
--tserver_master_addrs=127.4.228.254:45083
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.194
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:45:22.220124 28350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:45:22.221546 28350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:45:22.236977 28363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:22.236977 28360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:22.237469 28359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:45:22.238404 28350 server_base.cc:1034] running on GCE node
I20260810 18:45:22.239475 28350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:45:22.240728 28350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:45:22.241984 28350 hybrid_clock.cc:648] HybridClock initialized: now 1786387522241936 us; error 29 us; skew 500 ppm
I20260810 18:45:22.242396 28350 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:45:22 dist-test-slave-7kzw krb5kdc[26383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387522, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:45:22.246142 28350 init.cc:377] Logged in from keytab as kudu/127.4.228.194@KRBTEST.COM (short username kudu)
I20260810 18:45:22.247910 28350 webserver.cc:460] Webserver started at http://127.4.228.194:44003/ using document root <none> and password file <none>
I20260810 18:45:22.248680 28350 fs_manager.cc:359] Metadata directory not provided
I20260810 18:45:22.248816 28350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:45:22.254385 28350 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:45:22.257277 28373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:22.258860 28350 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 18:45:22.259057 28350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/wal
uuid: "108755ecae3544b8b7470da9de6965a6"
format_stamp: "Formatted at 2026-08-10 18:45:16 on dist-test-slave-7kzw"
server_key: "f05f8e081b4dc5adb1d2d5e8e46427a3"
server_key_iv: "2808b3b9383d60d98a14995237c418a6"
server_key_version: "encryptionkey@0"
I20260810 18:45:22.260051 28350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:45:22.275007 28350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:45:22.279682 28350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:45:22.279994 28350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:45:22.281118 28350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:45:22.282670 28350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:45:22.282812 28350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:22.282945 28350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:45:22.283044 28350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:22.319676 28350 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.194:44971
I20260810 18:45:22.319804 28544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.194:44971 every 8 connection(s)
I20260810 18:45:22.321261 28350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
I20260810 18:45:22.326086 28549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:22.324279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.194:35727)
0810 18:45:22.325003 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:22.325247 (+ 244us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":273,"threads_started":1}
W20260810 18:45:22.326889 28545 heartbeater.cc:644] Failed to heartbeat to 127.4.228.254:45083 (0 consecutive failures): Network error: Failed to ping master at 127.4.228.254:45083: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
I20260810 18:45:22.327788 28549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:22.327398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.194:39793)
0810 18:45:22.327588 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:22.327666 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260810 18:45:22.328744 28549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:22.328383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.194:59499)
0810 18:45:22.328533 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:22.328625 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260810 18:45:22.329079 28545 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 18:45:22.329571 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 28350
I20260810 18:45:22.330243 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.195:38771
--local_ip_for_outbound_sockets=127.4.228.195
--tserver_master_addrs=127.4.228.254:45083
--webserver_port=42761
--webserver_interface=127.4.228.195
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:40315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:45:22.688073 28550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:45:22.688382 28550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:45:22.688484 28550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:45:22.695896 28550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:45:22.696014 28550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:45:22.696128 28550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.195
I20260810 18:45:22.706055 28550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:40315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.195:38771
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.228.195
--webserver_port=42761
--tserver_master_addrs=127.4.228.254:45083
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.195
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:45:22.707496 28550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:45:22.708850 28550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:45:22.723033 28557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:22.722978 28560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:22.722978 28558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:45:22.724139 28550 server_base.cc:1034] running on GCE node
I20260810 18:45:22.724910 28550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:45:22.725991 28550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:45:22.727303 28550 hybrid_clock.cc:648] HybridClock initialized: now 1786387522727210 us; error 56 us; skew 500 ppm
I20260810 18:45:22.727743 28550 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:45:22 dist-test-slave-7kzw krb5kdc[26383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387522, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:45:22.734555 28550 init.cc:377] Logged in from keytab as kudu/127.4.228.195@KRBTEST.COM (short username kudu)
I20260810 18:45:22.736300 28550 webserver.cc:460] Webserver started at http://127.4.228.195:42761/ using document root <none> and password file <none>
I20260810 18:45:22.737094 28550 fs_manager.cc:359] Metadata directory not provided
I20260810 18:45:22.737251 28550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:45:22.743366 28550 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 18:45:22.746348 28571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:22.747969 28550 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 18:45:22.748178 28550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/wal
uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd"
format_stamp: "Formatted at 2026-08-10 18:45:16 on dist-test-slave-7kzw"
server_key: "fc452c4f5a67070abd032841211552a2"
server_key_iv: "79d0d0440dd80db90db48c342a8de42b"
server_key_version: "encryptionkey@0"
I20260810 18:45:22.749222 28550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:45:22.767143 28550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:45:22.771577 28550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:45:22.771883 28550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:45:22.772944 28550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:45:22.774353 28550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:45:22.774519 28550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:22.774653 28550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:45:22.774757 28550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:22.811120 28550 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.195:38771
I20260810 18:45:22.811226 28739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.195:38771 every 8 connection(s)
I20260810 18:45:22.812711 28550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
I20260810 18:45:22.818149 28743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:22.816205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.195:33301)
0810 18:45:22.817014 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:22.817246 (+ 232us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":301,"threads_started":1}
W20260810 18:45:22.819115 28741 heartbeater.cc:644] Failed to heartbeat to 127.4.228.254:45083 (0 consecutive failures): Network error: Failed to ping master at 127.4.228.254:45083: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
I20260810 18:45:22.820089 28743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:22.819733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.195:35225)
0810 18:45:22.819898 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:22.819971 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260810 18:45:22.821184 28743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:22.820852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.195:40639)
0810 18:45:22.821000 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:22.821075 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260810 18:45:22.821561 28741 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 18:45:22.822229 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 28550
I20260810 18:45:22.839100 28745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:22.838202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.193:56255)
0810 18:45:22.838705 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:22.838839 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":207,"threads_started":1}
I20260810 18:45:23.016727 28746 raft_consensus.cc:482] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:45:23.017104 28746 raft_consensus.cc:504] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6114ee077fd140739ff7a63019042a8f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42335 } } peers { permanent_uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38771 } } peers { permanent_uuid: "108755ecae3544b8b7470da9de6965a6" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44971 } }
I20260810 18:45:23.019141 28746 leader_election.cc:288] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a8e4f6f0df3e4d3c8890b77a53cce6dd (127.4.228.195:38771), 108755ecae3544b8b7470da9de6965a6 (127.4.228.194:44971)
Aug 10 18:45:23 dist-test-slave-7kzw krb5kdc[26383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387521, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for kudu/127.4.228.195@KRBTEST.COM
Aug 10 18:45:23 dist-test-slave-7kzw krb5kdc[26383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387521, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for kudu/127.4.228.194@KRBTEST.COM
I20260810 18:45:23.042840 28750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:23.019927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:39205 (local address 127.4.228.194:44971)
0810 18:45:23.022178 (+ 2251us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:23.022193 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:23.022231 (+ 38us) server_negotiation.cc:407] Connection header received
0810 18:45:23.023743 (+ 1512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:23.023752 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:23.024098 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:23.024413 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:23.026133 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:23.027413 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:23.029525 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:23.030185 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:23.030438 (+ 253us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:23.034877 (+ 4439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:23.034885 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:23.034898 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:23.034936 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:23.038069 (+ 3133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:23.039033 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:23.039041 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:23.039047 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:23.039292 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:23.039856 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:23.039861 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:23.039865 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:23.040595 (+ 730us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:23.040785 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:23.041920 (+ 1135us) server_negotiation.cc:299] Negotiation successful
0810 18:45:23.042352 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2064,"thread_start_us":1943,"threads_started":1}
I20260810 18:45:23.042846 28751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:23.019879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:41241 (local address 127.4.228.195:38771)
0810 18:45:23.021678 (+ 1799us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:23.021700 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:23.021759 (+ 59us) server_negotiation.cc:407] Connection header received
0810 18:45:23.023423 (+ 1664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:23.023433 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:23.023829 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:23.024201 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:23.026136 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:23.027427 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:23.029522 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:23.030192 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:23.030436 (+ 244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:23.034625 (+ 4189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:23.034634 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:23.034657 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:23.034696 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:23.037841 (+ 3145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:23.038799 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:23.038826 (+ 27us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:23.038835 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:23.039023 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:23.039576 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:23.039585 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:23.039590 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:23.040426 (+ 836us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:23.040633 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:23.041920 (+ 1287us) server_negotiation.cc:299] Negotiation successful
0810 18:45:23.042369 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1537,"thread_start_us":1290,"threads_started":1}
I20260810 18:45:23.042868 28745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:23.019548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:38771 (local address 127.4.228.193:41241)
0810 18:45:23.019876 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:23.019934 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:23.020606 (+ 672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:23.024312 (+ 3706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:23.024327 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:23.025199 (+ 872us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:45:23.025908 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:23.025930 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:23.027620 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:23.027625 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:23.029367 (+ 1742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:23.029381 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:23.030449 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:23.030485 (+ 36us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:23.030748 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:23.031664 (+ 916us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:45:23.031682 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:45:23.034250 (+ 2568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:45:23.038000 (+ 3750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:23.038011 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:23.038027 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:23.038615 (+ 588us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:23.039182 (+ 567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:23.039187 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:23.039191 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:23.039454 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:23.040639 (+ 1185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:45:23.040646 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:45:23.041210 (+ 564us) client_negotiation.cc:769] Sending connection context
0810 18:45:23.041639 (+ 429us) client_negotiation.cc:240] Negotiation successful
0810 18:45:23.042271 (+ 632us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":89,"spinlock_wait_cycles":8704}
I20260810 18:45:23.042878 28747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:23.019619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:44971 (local address 127.4.228.193:39205)
0810 18:45:23.021921 (+ 2302us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:23.021941 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:23.022084 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:23.024369 (+ 2285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:23.024374 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:23.025237 (+ 863us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:45:23.025906 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:23.025930 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:23.027627 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:23.027631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:23.029367 (+ 1736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:23.029381 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:23.030516 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:23.030525 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:23.030748 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:23.031623 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:45:23.031661 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:45:23.034593 (+ 2932us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:45:23.038271 (+ 3678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:23.038277 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:23.038282 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:23.038863 (+ 581us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:23.039461 (+ 598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:23.039466 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:23.039469 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:23.039706 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:23.040752 (+ 1046us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:45:23.040757 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:45:23.041210 (+ 453us) client_negotiation.cc:769] Sending connection context
0810 18:45:23.041617 (+ 407us) client_negotiation.cc:240] Negotiation successful
0810 18:45:23.042421 (+ 804us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2136,"mutex_wait_us":431,"spinlock_wait_cycles":42752,"thread_start_us":1946,"threads_started":1}
W20260810 18:45:23.046662 28173 leader_election.cc:341] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 108755ecae3544b8b7470da9de6965a6 (127.4.228.194:44971): Not found: Tablet not found: 68525487e52345708d37cb880fb88a5d
W20260810 18:45:23.046940 28175 leader_election.cc:341] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a8e4f6f0df3e4d3c8890b77a53cce6dd (127.4.228.195:38771): Not found: Tablet not found: 68525487e52345708d37cb880fb88a5d
I20260810 18:45:23.047137 28175 leader_election.cc:302] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6114ee077fd140739ff7a63019042a8f; no voters: 108755ecae3544b8b7470da9de6965a6, a8e4f6f0df3e4d3c8890b77a53cce6dd
I20260810 18:45:23.047652 28746 raft_consensus.cc:2726] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 18:45:23.330667 28752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:23.329698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.194:60643)
0810 18:45:23.330188 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:23.330345 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":227,"threads_started":1}
I20260810 18:45:23.823012 28753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:23.822103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.195:50151)
0810 18:45:23.822674 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:23.822799 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":241,"threads_started":1}
I20260810 18:45:23.840726 28754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:23.839967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.193:36213)
0810 18:45:23.840491 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:23.840571 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":305,"threads_started":1}
I20260810 18:45:24.332316 28755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:24.331562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.194:35235)
0810 18:45:24.332058 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:24.332147 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":271,"threads_started":1}
I20260810 18:45:24.824903 28758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:24.823988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.195:53557)
0810 18:45:24.824609 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:24.824706 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":272,"threads_started":1}
I20260810 18:45:24.842695 28760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:24.841804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.193:51731)
0810 18:45:24.842320 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:24.842404 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":259,"threads_started":1}
I20260810 18:45:25.276324 28761 raft_consensus.cc:482] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:45:25.276603 28761 raft_consensus.cc:504] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6114ee077fd140739ff7a63019042a8f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42335 } } peers { permanent_uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38771 } } peers { permanent_uuid: "108755ecae3544b8b7470da9de6965a6" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44971 } }
I20260810 18:45:25.277417 28761 leader_election.cc:288] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a8e4f6f0df3e4d3c8890b77a53cce6dd (127.4.228.195:38771), 108755ecae3544b8b7470da9de6965a6 (127.4.228.194:44971)
W20260810 18:45:25.278671 28175 leader_election.cc:341] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a8e4f6f0df3e4d3c8890b77a53cce6dd (127.4.228.195:38771): Not found: Tablet not found: 68525487e52345708d37cb880fb88a5d
W20260810 18:45:25.278893 28173 leader_election.cc:341] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 108755ecae3544b8b7470da9de6965a6 (127.4.228.194:44971): Not found: Tablet not found: 68525487e52345708d37cb880fb88a5d
I20260810 18:45:25.279067 28173 leader_election.cc:302] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6114ee077fd140739ff7a63019042a8f; no voters: 108755ecae3544b8b7470da9de6965a6, a8e4f6f0df3e4d3c8890b77a53cce6dd
I20260810 18:45:25.279382 28761 raft_consensus.cc:2726] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 18:45:25.334146 28764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:25.333308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.194:39171)
0810 18:45:25.333852 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:25.333948 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":288,"threads_started":1}
I20260810 18:45:25.826856 28765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:25.825883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.195:56099)
0810 18:45:25.826395 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:25.826580 (+ 185us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":275,"threads_started":1}
I20260810 18:45:25.844559 28767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:25.843657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.193:55671)
0810 18:45:25.844155 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:25.844302 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":280,"threads_started":1}
I20260810 18:45:26.335855 28769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:26.335142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.194:32959)
0810 18:45:26.335574 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:26.335660 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":210,"threads_started":1}
I20260810 18:45:26.828645 28770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:26.827871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.195:35941)
0810 18:45:26.828374 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:26.828469 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":265,"threads_started":1}
I20260810 18:45:26.846719 28771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:26.845774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.193:40917)
0810 18:45:26.846292 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:26.846383 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":258,"threads_started":1}
I20260810 18:45:26.849383 28773 raft_consensus.cc:482] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:45:26.849588 28773 raft_consensus.cc:504] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6114ee077fd140739ff7a63019042a8f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42335 } } peers { permanent_uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38771 } } peers { permanent_uuid: "108755ecae3544b8b7470da9de6965a6" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44971 } }
I20260810 18:45:26.850314 28773 leader_election.cc:288] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a8e4f6f0df3e4d3c8890b77a53cce6dd (127.4.228.195:38771), 108755ecae3544b8b7470da9de6965a6 (127.4.228.194:44971)
W20260810 18:45:26.851224 28175 leader_election.cc:341] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a8e4f6f0df3e4d3c8890b77a53cce6dd (127.4.228.195:38771): Not found: Tablet not found: 68525487e52345708d37cb880fb88a5d
W20260810 18:45:26.851408 28173 leader_election.cc:341] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 108755ecae3544b8b7470da9de6965a6 (127.4.228.194:44971): Not found: Tablet not found: 68525487e52345708d37cb880fb88a5d
I20260810 18:45:26.851570 28173 leader_election.cc:302] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6114ee077fd140739ff7a63019042a8f; no voters: 108755ecae3544b8b7470da9de6965a6, a8e4f6f0df3e4d3c8890b77a53cce6dd
I20260810 18:45:26.851886 28773 raft_consensus.cc:2726] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 18:45:27.337589 28774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:27.336841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.194:51893)
0810 18:45:27.337327 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:27.337414 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":251,"threads_started":1}
I20260810 18:45:27.830657 28775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:27.829760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.195:45661)
0810 18:45:27.830316 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:27.830407 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":306,"threads_started":1}
I20260810 18:45:27.848440 28776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:27.847707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.193:46735)
0810 18:45:27.848164 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:27.848255 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":234,"threads_started":1}
I20260810 18:45:28.339438 28777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:28.338593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.194:40577)
0810 18:45:28.339119 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:28.339213 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":286,"threads_started":1}
I20260810 18:45:28.832430 28778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:28.831600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.195:44577)
0810 18:45:28.832160 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:28.832243 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":313,"threads_started":1}
I20260810 18:45:28.850348 28779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:28.849480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.193:42123)
0810 18:45:28.850043 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:28.850147 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":315,"threads_started":1}
I20260810 18:45:29.341379 28780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:29.340499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.194:37389)
0810 18:45:29.341094 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:29.341184 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":300,"threads_started":1}
I20260810 18:45:29.834334 28785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:29.833501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.195:47999)
0810 18:45:29.834026 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:29.834111 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":249,"threads_started":1}
I20260810 18:45:29.852106 28788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:29.851340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.193:55035)
0810 18:45:29.851831 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:29.851924 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":243,"threads_started":1}
I20260810 18:45:30.343230 28789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:30.342409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.194:53341)
0810 18:45:30.342939 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:30.343027 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":309,"threads_started":1}
I20260810 18:45:30.836184 28790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:30.835385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.195:46647)
0810 18:45:30.835892 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:30.835983 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":283,"threads_started":1}
I20260810 18:45:30.853876 28791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:30.853037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.193:36061)
0810 18:45:30.853535 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:30.853626 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":292,"threads_started":1}
I20260810 18:45:31.345285 28793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:31.344305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.194:32915)
0810 18:45:31.345008 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:31.345112 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":415,"threads_started":1}
I20260810 18:45:31.557260 28794 raft_consensus.cc:482] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:45:31.557510 28794 raft_consensus.cc:504] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6114ee077fd140739ff7a63019042a8f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42335 } } peers { permanent_uuid: "a8e4f6f0df3e4d3c8890b77a53cce6dd" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 38771 } } peers { permanent_uuid: "108755ecae3544b8b7470da9de6965a6" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 44971 } }
I20260810 18:45:31.558321 28794 leader_election.cc:288] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a8e4f6f0df3e4d3c8890b77a53cce6dd (127.4.228.195:38771), 108755ecae3544b8b7470da9de6965a6 (127.4.228.194:44971)
W20260810 18:45:31.559443 28175 leader_election.cc:341] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a8e4f6f0df3e4d3c8890b77a53cce6dd (127.4.228.195:38771): Not found: Tablet not found: 68525487e52345708d37cb880fb88a5d
W20260810 18:45:31.559679 28173 leader_election.cc:341] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 108755ecae3544b8b7470da9de6965a6 (127.4.228.194:44971): Not found: Tablet not found: 68525487e52345708d37cb880fb88a5d
I20260810 18:45:31.559868 28173 leader_election.cc:302] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6114ee077fd140739ff7a63019042a8f; no voters: 108755ecae3544b8b7470da9de6965a6, a8e4f6f0df3e4d3c8890b77a53cce6dd
I20260810 18:45:31.560194 28794 raft_consensus.cc:2726] T 68525487e52345708d37cb880fb88a5d P 6114ee077fd140739ff7a63019042a8f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 18:45:31.838013 28799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:31.837240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.195:46595)
0810 18:45:31.837769 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:31.837852 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":272,"threads_started":1}
I20260810 18:45:31.855849 28800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:31.854988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.193:51911)
0810 18:45:31.855516 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:31.855651 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":260,"threads_started":1}
I20260810 18:45:32.347256 28803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:32.346393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.194:60457)
0810 18:45:32.346959 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:32.347054 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":339,"threads_started":1}
I20260810 18:45:32.839785 28804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:32.839031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.195:51129)
0810 18:45:32.839490 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:32.839570 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":230,"threads_started":1}
I20260810 18:45:32.857686 28805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:32.856899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.193:39587)
0810 18:45:32.857387 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:32.857469 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":273,"threads_started":1}
I20260810 18:45:33.349165 28806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:33.348331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.194:58759)
0810 18:45:33.348873 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:33.348976 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":257,"threads_started":1}
I20260810 18:45:33.841578 28807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:33.840745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.195:50893)
0810 18:45:33.841294 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:33.841389 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":273,"threads_started":1}
I20260810 18:45:33.859362 28808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:33.858620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.193:50435)
0810 18:45:33.859079 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:33.859179 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":247,"threads_started":1}
I20260810 18:45:34.351191 28809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:34.350295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.194:53639)
0810 18:45:34.350886 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:34.350983 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":336,"threads_started":1}
I20260810 18:45:34.843377 28813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:34.842584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.195:60425)
0810 18:45:34.843087 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:34.843197 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":259,"threads_started":1}
I20260810 18:45:34.861119 28814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:34.860345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.193:34989)
0810 18:45:34.860838 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:34.860939 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":271,"threads_started":1}
I20260810 18:45:35.353354 28817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:35.352397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.194:35121)
0810 18:45:35.353067 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:35.353169 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":410,"threads_started":1}
I20260810 18:45:35.845480 28819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:35.844496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.195:54031)
0810 18:45:35.845122 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:35.845234 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":348,"threads_started":1}
I20260810 18:45:35.863005 28820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:35.862106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.193:45111)
0810 18:45:35.862687 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:35.862789 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":260,"threads_started":1}
I20260810 18:45:36.355557 28822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:36.354619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.194:55063)
0810 18:45:36.355212 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:36.355307 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":323,"threads_started":1}
I20260810 18:45:36.847481 28823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:36.846621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.195:53875)
0810 18:45:36.847138 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:36.847243 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":269,"threads_started":1}
I20260810 18:45:36.865084 28824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:36.864273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.193:55763)
0810 18:45:36.864814 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:36.864907 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":300,"threads_started":1}
I20260810 18:45:37.357806 28825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:37.356902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.194:48751)
0810 18:45:37.357450 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:37.357592 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":289,"threads_started":1}
I20260810 18:45:37.833175 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 28089
I20260810 18:45:37.849779 28827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:37.848777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.195:35899)
0810 18:45:37.849379 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:37.849551 (+ 172us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":329,"threads_started":1}
I20260810 18:45:37.851708 28826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:37.834402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49510 (local address 127.4.228.193:42335)
0810 18:45:37.835196 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:37.835212 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:37.835499 (+ 287us) server_negotiation.cc:407] Connection header received
0810 18:45:37.835684 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:37.835693 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:37.835925 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:37.836076 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:37.837846 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:37.839047 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:37.840693 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:37.841358 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:37.841491 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:37.844747 (+ 3256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:37.844760 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:37.844777 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:37.844820 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:37.848109 (+ 3289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:37.849027 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:37.849041 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:37.849047 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:37.849252 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:37.849825 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:37.849830 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:37.849832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:37.850329 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:37.850682 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:37.851074 (+ 392us) server_negotiation.cc:299] Negotiation successful
0810 18:45:37.851307 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":366,"threads_started":1}
I20260810 18:45:37.853334 28338 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:45:37.946004 28830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:37.944782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:45083 (local address 127.4.228.193:44309)
0810 18:45:37.945355 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:37.945546 (+ 191us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:45083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":283,"threads_started":1}
I20260810 18:45:37.946014 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 28089
I20260810 18:45:37.979499 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 28350
I20260810 18:45:37.998993 28831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:37.980536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51126 (local address 127.4.228.194:44971)
0810 18:45:37.981036 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:37.981041 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:37.981059 (+ 18us) server_negotiation.cc:407] Connection header received
0810 18:45:37.981120 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:37.981124 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:37.981187 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:37.981298 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:37.982762 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:37.983633 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:37.984834 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:37.985313 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:37.985425 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:37.992985 (+ 7560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:37.992990 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:37.992994 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:37.993036 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:37.995968 (+ 2932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:37.996736 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:37.996741 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:37.996744 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:37.996914 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:37.997302 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:37.997305 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:37.997308 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:37.997619 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:37.997881 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:37.998168 (+ 287us) server_negotiation.cc:299] Negotiation successful
0810 18:45:37.998367 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":274,"threads_started":1}
I20260810 18:45:37.999850 28538 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:45:38.086642 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 28350
I20260810 18:45:38.115046 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 28550
I20260810 18:45:38.129325 28836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:38.115983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44848 (local address 127.4.228.195:38771)
0810 18:45:38.116632 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:38.116637 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:38.116656 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:45:38.116718 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:38.116722 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:38.116794 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:38.116890 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:38.118333 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:38.119179 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:38.120434 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:38.120859 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:38.120958 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:38.123949 (+ 2991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:38.123954 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:38.123958 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:38.123992 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:38.126829 (+ 2837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:38.127534 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:38.127539 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:38.127541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:38.127748 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:38.128096 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:38.128099 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:38.128101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:38.128367 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:38.128584 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:38.128846 (+ 262us) server_negotiation.cc:299] Negotiation successful
0810 18:45:38.129016 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":372,"threads_started":1}
I20260810 18:45:38.130059 28734 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:45:38.214102 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 28550
2026-08-10T18:45:38Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25400 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:45:38 dist-test-slave-7kzw krb5kdc[28843](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:45:38 dist-test-slave-7kzw krb5kdc[28843](info): set up 2 sockets
Aug 10 18:45:38 dist-test-slave-7kzw krb5kdc[28843](info): commencing operation
krb5kdc: starting...
W20260810 18:45:39.598863 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.329s user 0.002s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:45:39 dist-test-slave-7kzw krb5kdc[28843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:45:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:45:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.228.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
I20260810 18:45:39.922943 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:35121
--webserver_interface=127.4.228.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:34661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:35121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:45:40.290740 28862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:45:40.291136 28862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:45:40.291303 28862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:45:40.299626 28862 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 18:45:40.299778 28862 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 18:45:40.299850 28862 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 18:45:40.299947 28862 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:45:40.300029 28862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:45:40.300158 28862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:45:40.300248 28862 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:45:40.300330 28862 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:45:40.310922 28862 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:34661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:35121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:35121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:45:40.312424 28862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:45:40.313932 28862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:45:40.327080 28871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:40.327167 28868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:40.327358 28867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:45:40.328370 28862 server_base.cc:1034] running on GCE node
I20260810 18:45:40.329208 28862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:45:40.330742 28862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:45:40.331992 28862 hybrid_clock.cc:648] HybridClock initialized: now 1786387540331929 us; error 43 us; skew 500 ppm
I20260810 18:45:40.332347 28862 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:45:40 dist-test-slave-7kzw krb5kdc[28843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387540, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:45:40.335886 28862 init.cc:377] Logged in from keytab as kudu/127.4.228.254@KRBTEST.COM (short username kudu)
I20260810 18:45:40.337508 28862 webserver.cc:460] Webserver started at http://127.4.228.254:34223/ using document root <none> and password file <none>
I20260810 18:45:40.338249 28862 fs_manager.cc:359] Metadata directory not provided
I20260810 18:45:40.338387 28862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:45:40.338759 28862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:45:40.341804 28862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "4d47ce6d53164cc6b711addede1613ca"
format_stamp: "Formatted at 2026-08-10 18:45:40 on dist-test-slave-7kzw"
server_key: "e80ea1563709bcc7d4ff985586949483"
server_key_iv: "47a9842fb3eff52a620674c42de1c8fe"
server_key_version: "encryptionkey@0"
I20260810 18:45:40.342675 28862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "4d47ce6d53164cc6b711addede1613ca"
format_stamp: "Formatted at 2026-08-10 18:45:40 on dist-test-slave-7kzw"
server_key: "e80ea1563709bcc7d4ff985586949483"
server_key_iv: "47a9842fb3eff52a620674c42de1c8fe"
server_key_version: "encryptionkey@0"
I20260810 18:45:40.347895 28862 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 18:45:40.351475 28882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:40.352919 28862 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260810 18:45:40.353120 28862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "4d47ce6d53164cc6b711addede1613ca"
format_stamp: "Formatted at 2026-08-10 18:45:40 on dist-test-slave-7kzw"
server_key: "e80ea1563709bcc7d4ff985586949483"
server_key_iv: "47a9842fb3eff52a620674c42de1c8fe"
server_key_version: "encryptionkey@0"
I20260810 18:45:40.353343 28862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:45:40.375419 28862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:45:40.379951 28862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:45:40.380246 28862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:45:40.400731 28862 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:35121
I20260810 18:45:40.400756 28956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:35121 every 8 connection(s)
I20260810 18:45:40.402179 28862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:45:40.405447 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 28862
I20260810 18:45:40.405702 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:45:40.406687 5011 external_mini_cluster.cc:1428] Setting key c2248b7c1d2396edfed5b27facbebea9
I20260810 18:45:40.406978 28957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:45:40.423398 28957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca: Bootstrap starting.
Aug 10 18:45:40 dist-test-slave-7kzw krb5kdc[28843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:45:40.427093 28957 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca: Neither blocks nor log segments found. Creating new log.
I20260810 18:45:40.428246 28957 log.cc:824] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca: Log is configured to *not* fsync() on all Append() calls
I20260810 18:45:40.431476 28957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca: No bootstrap required, opened a new log
I20260810 18:45:40.433923 28961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:40.409133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49858 (local address 127.4.228.254:35121)
0810 18:45:40.410130 (+ 997us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:40.410143 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:40.410203 (+ 60us) server_negotiation.cc:407] Connection header received
0810 18:45:40.412459 (+ 2256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:40.412487 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:40.412961 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:40.413376 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:40.415595 (+ 2219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:40.417016 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:40.419095 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:40.419842 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:40.419995 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:40.425321 (+ 5326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:40.425330 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:40.425346 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:40.425389 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:40.429240 (+ 3851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:40.430079 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:40.430087 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:40.430094 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:40.430283 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:40.430784 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:40.430789 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:40.430791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:40.431371 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:40.431744 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:40.432271 (+ 527us) server_negotiation.cc:299] Negotiation successful
0810 18:45:40.432782 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":734,"thread_start_us":455,"threads_started":1}
I20260810 18:45:40.439929 28957 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d47ce6d53164cc6b711addede1613ca" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35121 } }
I20260810 18:45:40.440310 28957 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:45:40.440423 28957 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d47ce6d53164cc6b711addede1613ca, State: Initialized, Role: FOLLOWER
I20260810 18:45:40.440999 28957 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d47ce6d53164cc6b711addede1613ca" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35121 } }
I20260810 18:45:40.441270 28957 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:45:40.441426 28957 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:45:40.441614 28957 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:45:40.444239 28957 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d47ce6d53164cc6b711addede1613ca" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35121 } }
I20260810 18:45:40.444885 28957 leader_election.cc:302] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d47ce6d53164cc6b711addede1613ca; no voters:
I20260810 18:45:40.445528 28957 leader_election.cc:288] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:45:40.445686 28963 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:45:40.447311 28963 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [term 1 LEADER]: Becoming Leader. State: Replica: 4d47ce6d53164cc6b711addede1613ca, State: Running, Role: LEADER
I20260810 18:45:40.447881 28963 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d47ce6d53164cc6b711addede1613ca" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35121 } }
I20260810 18:45:40.448069 28957 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:45:40.451121 28967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d47ce6d53164cc6b711addede1613ca. Latest consensus state: current_term: 1 leader_uuid: "4d47ce6d53164cc6b711addede1613ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d47ce6d53164cc6b711addede1613ca" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35121 } } }
I20260810 18:45:40.451598 28967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [sys.catalog]: This master's current role is: LEADER
I20260810 18:45:40.451819 28966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d47ce6d53164cc6b711addede1613ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d47ce6d53164cc6b711addede1613ca" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35121 } } }
I20260810 18:45:40.452198 28966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [sys.catalog]: This master's current role is: LEADER
I20260810 18:45:40.454231 28975 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:45:40.459578 28975 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:45:40.469686 28975 catalog_manager.cc:1347] Generated new cluster ID: f39befd62fef402e98e6aef83721c785
I20260810 18:45:40.469826 28975 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:45:40.497197 28975 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:45:40.498765 28975 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:45:40.508004 28975 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca: Generated new TSK 0
I20260810 18:45:40.508924 28975 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.228.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab.
Entry for principal kudu/127.4.228.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab.
I20260810 18:45:40.595851 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.193:0
--local_ip_for_outbound_sockets=127.4.228.193
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:35121
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:34661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:45:40.988763 28994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:45:40.989140 28994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:45:40.989310 28994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:45:40.996677 28994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:45:40.996852 28994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:45:40.997027 28994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.193
I20260810 18:45:41.007449 28994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:34661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:35121
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.193
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:45:41.008932 28994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:45:41.010573 28994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:45:41.024727 29002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:41.024752 29004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:41.025408 29001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:45:41.026240 28994 server_base.cc:1034] running on GCE node
I20260810 18:45:41.026928 28994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:45:41.027899 28994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:45:41.029125 28994 hybrid_clock.cc:648] HybridClock initialized: now 1786387541029049 us; error 58 us; skew 500 ppm
I20260810 18:45:41.029464 28994 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:45:41 dist-test-slave-7kzw krb5kdc[28843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387541, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:45:41.032951 28994 init.cc:377] Logged in from keytab as kudu/127.4.228.193@KRBTEST.COM (short username kudu)
I20260810 18:45:41.034631 28994 webserver.cc:460] Webserver started at http://127.4.228.193:43447/ using document root <none> and password file <none>
I20260810 18:45:41.035382 28994 fs_manager.cc:359] Metadata directory not provided
I20260810 18:45:41.035547 28994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:45:41.035887 28994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:45:41.038961 28994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/data/instance:
uuid: "3864ce298e8744258f3cdc6c68f97864"
format_stamp: "Formatted at 2026-08-10 18:45:41 on dist-test-slave-7kzw"
server_key: "76cb203db86fc72e217f15ef5039d2a9"
server_key_iv: "d84ca856c058ce9d594105efeb53255e"
server_key_version: "encryptionkey@0"
I20260810 18:45:41.039776 28994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance:
uuid: "3864ce298e8744258f3cdc6c68f97864"
format_stamp: "Formatted at 2026-08-10 18:45:41 on dist-test-slave-7kzw"
server_key: "76cb203db86fc72e217f15ef5039d2a9"
server_key_iv: "d84ca856c058ce9d594105efeb53255e"
server_key_version: "encryptionkey@0"
I20260810 18:45:41.045017 28994 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 18:45:41.048957 29015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:41.050593 28994 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260810 18:45:41.050807 28994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/wal
uuid: "3864ce298e8744258f3cdc6c68f97864"
format_stamp: "Formatted at 2026-08-10 18:45:41 on dist-test-slave-7kzw"
server_key: "76cb203db86fc72e217f15ef5039d2a9"
server_key_iv: "d84ca856c058ce9d594105efeb53255e"
server_key_version: "encryptionkey@0"
I20260810 18:45:41.051051 28994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:45:41.085950 28994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:45:41.091007 28994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:45:41.091439 28994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:45:41.092583 28994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:45:41.094108 28994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:45:41.094225 28994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:41.094396 28994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:45:41.094508 28994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:41.133131 28994 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.193:39905
I20260810 18:45:41.133188 29196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.193:39905 every 8 connection(s)
I20260810 18:45:41.134747 28994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
I20260810 18:45:41.142714 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 28994
I20260810 18:45:41.142953 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance
I20260810 18:45:41.143666 5011 external_mini_cluster.cc:1428] Setting key 5ce10a179245ed040b553fc57a13f883
Aug 10 18:45:41 dist-test-slave-7kzw krb5kdc[28843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387541, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:45:41.156944 29200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:41.137671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:54883 (local address 127.4.228.254:35121)
0810 18:45:41.138166 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:41.138174 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:41.139263 (+ 1089us) server_negotiation.cc:407] Connection header received
0810 18:45:41.141109 (+ 1846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:41.141113 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:41.141201 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:41.141303 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:41.142907 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:41.143685 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:41.145513 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:41.146051 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:41.146171 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:41.150707 (+ 4536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:41.150712 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:41.150716 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:41.150750 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:41.153452 (+ 2702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:41.154222 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:41.154235 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:41.154238 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:41.154371 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:41.154953 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:41.154957 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:41.154959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:41.155239 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:41.155418 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:41.156303 (+ 885us) server_negotiation.cc:299] Negotiation successful
0810 18:45:41.156545 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":224,"threads_started":1}
I20260810 18:45:41.157966 29201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:41.138301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.193:54883)
0810 18:45:41.139089 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:41.139136 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:41.140836 (+ 1700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:41.141505 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:41.141513 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:41.142069 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:45:41.142720 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:41.142740 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:41.143869 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:41.143874 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:41.145282 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:41.145296 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:41.146310 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:41.146319 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:41.146665 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:41.147409 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:45:41.147445 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:45:41.150432 (+ 2987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:45:41.153613 (+ 3181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:41.153623 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:41.153641 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:41.154077 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:41.154523 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:41.154528 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:41.154532 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:41.154814 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:41.155380 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:45:41.155388 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:45:41.155805 (+ 417us) client_negotiation.cc:769] Sending connection context
0810 18:45:41.156181 (+ 376us) client_negotiation.cc:240] Negotiation successful
0810 18:45:41.156730 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":356,"threads_started":1}
I20260810 18:45:41.159598 29198 heartbeater.cc:347] Connected to a master server at 127.4.228.254:35121
I20260810 18:45:41.159935 29198 heartbeater.cc:464] Registering TS with master...
I20260810 18:45:41.160837 29198 heartbeater.cc:511] Master 127.4.228.254:35121 requested a full tablet report, sending...
I20260810 18:45:41.163271 28902 ts_manager.cc:194] Registered new tserver with Master: 3864ce298e8744258f3cdc6c68f97864 (127.4.228.193:39905)
I20260810 18:45:41.164840 28902 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.193@KRBTEST.COM'} at 127.4.228.193:54883
WARNING: no policy specified for kudu/127.4.228.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab.
Entry for principal kudu/127.4.228.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab.
I20260810 18:45:41.214318 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.194:0
--local_ip_for_outbound_sockets=127.4.228.194
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:35121
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:34661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:45:41.571034 29207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:45:41.571348 29207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:45:41.571448 29207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:45:41.578752 29207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:45:41.578863 29207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:45:41.578976 29207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.194
I20260810 18:45:41.588748 29207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:34661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:35121
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.194
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:45:41.590008 29207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:45:41.591355 29207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:45:41.605406 29214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:41.605408 29217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:41.605440 29219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:45:41.606153 29207 server_base.cc:1034] running on GCE node
I20260810 18:45:41.606870 29207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:45:41.607870 29207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:45:41.609109 29207 hybrid_clock.cc:648] HybridClock initialized: now 1786387541609053 us; error 54 us; skew 500 ppm
I20260810 18:45:41.609440 29207 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:45:41 dist-test-slave-7kzw krb5kdc[28843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387541, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:45:41.612756 29207 init.cc:377] Logged in from keytab as kudu/127.4.228.194@KRBTEST.COM (short username kudu)
I20260810 18:45:41.614311 29207 webserver.cc:460] Webserver started at http://127.4.228.194:37951/ using document root <none> and password file <none>
I20260810 18:45:41.615108 29207 fs_manager.cc:359] Metadata directory not provided
I20260810 18:45:41.615269 29207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:45:41.615609 29207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:45:41.618616 29207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/data/instance:
uuid: "c04280931825462796891c073e8074ac"
format_stamp: "Formatted at 2026-08-10 18:45:41 on dist-test-slave-7kzw"
server_key: "3633085410b20a64de2bd071215e3de3"
server_key_iv: "1e79c438679ed9c0df5d9d902c593543"
server_key_version: "encryptionkey@0"
I20260810 18:45:41.619385 29207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance:
uuid: "c04280931825462796891c073e8074ac"
format_stamp: "Formatted at 2026-08-10 18:45:41 on dist-test-slave-7kzw"
server_key: "3633085410b20a64de2bd071215e3de3"
server_key_iv: "1e79c438679ed9c0df5d9d902c593543"
server_key_version: "encryptionkey@0"
I20260810 18:45:41.624791 29207 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 18:45:41.628361 29229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:41.629666 29207 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:45:41.629850 29207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/wal
uuid: "c04280931825462796891c073e8074ac"
format_stamp: "Formatted at 2026-08-10 18:45:41 on dist-test-slave-7kzw"
server_key: "3633085410b20a64de2bd071215e3de3"
server_key_iv: "1e79c438679ed9c0df5d9d902c593543"
server_key_version: "encryptionkey@0"
I20260810 18:45:41.630054 29207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:45:41.643778 29207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:45:41.648571 29207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:45:41.648921 29207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:45:41.650076 29207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:45:41.651648 29207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:45:41.651764 29207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:41.651928 29207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:45:41.652012 29207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:41.697173 29207 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.194:45331
I20260810 18:45:41.697234 29395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.194:45331 every 8 connection(s)
I20260810 18:45:41.698566 29207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
I20260810 18:45:41.706629 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 29207
I20260810 18:45:41.706914 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance
I20260810 18:45:41.707418 5011 external_mini_cluster.cc:1428] Setting key 1c19227e3a98204ef401fa5b0b7417c9
Aug 10 18:45:41 dist-test-slave-7kzw krb5kdc[28843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387541, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:45:41.720882 29401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:41.701393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:33789 (local address 127.4.228.254:35121)
0810 18:45:41.701886 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:41.701891 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:41.702861 (+ 970us) server_negotiation.cc:407] Connection header received
0810 18:45:41.704714 (+ 1853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:41.704721 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:41.704821 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:41.704948 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:41.706846 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:41.707766 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:41.709740 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:41.710180 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:41.710322 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:41.714269 (+ 3947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:41.714275 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:41.714278 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:41.714314 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:41.717099 (+ 2785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:41.717920 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:41.717925 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:41.717927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:41.718068 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:41.718576 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:41.718579 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:41.718581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:41.718995 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:41.719185 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:41.720220 (+ 1035us) server_negotiation.cc:299] Negotiation successful
0810 18:45:41.720459 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":277,"threads_started":1}
I20260810 18:45:41.721915 29402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:41.701882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.194:33789)
0810 18:45:41.702693 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:41.702738 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:41.704415 (+ 1677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:41.705214 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:41.705222 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:41.705765 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:45:41.706568 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:41.706600 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:41.707945 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:41.707950 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:41.709579 (+ 1629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:41.709591 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:41.710422 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:41.710431 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:41.710718 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:41.711472 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:45:41.711496 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:45:41.714021 (+ 2525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:45:41.717271 (+ 3250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:41.717282 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:41.717299 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:41.717774 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:41.718190 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:41.718195 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:41.718199 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:41.718400 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:41.719278 (+ 878us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:45:41.719289 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:45:41.719814 (+ 525us) client_negotiation.cc:769] Sending connection context
0810 18:45:41.720115 (+ 301us) client_negotiation.cc:240] Negotiation successful
0810 18:45:41.720687 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":293,"threads_started":1}
I20260810 18:45:41.723574 29396 heartbeater.cc:347] Connected to a master server at 127.4.228.254:35121
I20260810 18:45:41.723974 29396 heartbeater.cc:464] Registering TS with master...
I20260810 18:45:41.724908 29396 heartbeater.cc:511] Master 127.4.228.254:35121 requested a full tablet report, sending...
I20260810 18:45:41.726665 28902 ts_manager.cc:194] Registered new tserver with Master: c04280931825462796891c073e8074ac (127.4.228.194:45331)
I20260810 18:45:41.727551 28902 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.194@KRBTEST.COM'} at 127.4.228.194:33789
WARNING: no policy specified for kudu/127.4.228.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab.
Entry for principal kudu/127.4.228.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab.
I20260810 18:45:41.779255 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.195:0
--local_ip_for_outbound_sockets=127.4.228.195
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:35121
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:34661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:45:42.138859 29408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:45:42.139240 29408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:45:42.139406 29408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:45:42.146924 29408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:45:42.147034 29408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:45:42.147146 29408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.195
I20260810 18:45:42.157270 29408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:34661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:35121
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.195
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:45:42.158787 29408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:45:42.160130 29408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 18:45:42.168730 29198 heartbeater.cc:503] Master 127.4.228.254:35121 was elected leader, sending a full tablet report...
W20260810 18:45:42.176776 29416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:42.176903 29418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:45:42.176878 29414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:45:42.177649 29408 server_base.cc:1034] running on GCE node
I20260810 18:45:42.178316 29408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:45:42.179455 29408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:45:42.180701 29408 hybrid_clock.cc:648] HybridClock initialized: now 1786387542180610 us; error 63 us; skew 500 ppm
I20260810 18:45:42.181088 29408 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:45:42 dist-test-slave-7kzw krb5kdc[28843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387542, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:45:42.184818 29408 init.cc:377] Logged in from keytab as kudu/127.4.228.195@KRBTEST.COM (short username kudu)
I20260810 18:45:42.186647 29408 webserver.cc:460] Webserver started at http://127.4.228.195:42191/ using document root <none> and password file <none>
I20260810 18:45:42.187480 29408 fs_manager.cc:359] Metadata directory not provided
I20260810 18:45:42.187656 29408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:45:42.188025 29408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:45:42.191414 29408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/data/instance:
uuid: "23521ea509214ca89745701374ff8eda"
format_stamp: "Formatted at 2026-08-10 18:45:42 on dist-test-slave-7kzw"
server_key: "8d48b38acf11af152cf35838bc180760"
server_key_iv: "912a5d8864142603f986f7f3dcf7d872"
server_key_version: "encryptionkey@0"
I20260810 18:45:42.192307 29408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance:
uuid: "23521ea509214ca89745701374ff8eda"
format_stamp: "Formatted at 2026-08-10 18:45:42 on dist-test-slave-7kzw"
server_key: "8d48b38acf11af152cf35838bc180760"
server_key_iv: "912a5d8864142603f986f7f3dcf7d872"
server_key_version: "encryptionkey@0"
I20260810 18:45:42.198261 29408 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 18:45:42.202529 29426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:42.204047 29408 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:45:42.204284 29408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/wal
uuid: "23521ea509214ca89745701374ff8eda"
format_stamp: "Formatted at 2026-08-10 18:45:42 on dist-test-slave-7kzw"
server_key: "8d48b38acf11af152cf35838bc180760"
server_key_iv: "912a5d8864142603f986f7f3dcf7d872"
server_key_version: "encryptionkey@0"
I20260810 18:45:42.204514 29408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:45:42.239765 29408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:45:42.244491 29408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:45:42.244817 29408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:45:42.245864 29408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:45:42.247407 29408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:45:42.247515 29408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:45:42.247684 29408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:45:42.247805 29408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260810 18:45:42.285519 29408 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.195:40527
I20260810 18:45:42.285568 29586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.195:40527 every 8 connection(s)
I20260810 18:45:42.286947 29408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
I20260810 18:45:42.292289 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 29408
I20260810 18:45:42.292568 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance
I20260810 18:45:42.293145 5011 external_mini_cluster.cc:1428] Setting key a76299a0e53b853f06d9721296322d4a
Aug 10 18:45:42 dist-test-slave-7kzw krb5kdc[28843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387542, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:45:42.309859 29591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:42.289823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:37633 (local address 127.4.228.254:35121)
0810 18:45:42.290380 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:42.290385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:42.291578 (+ 1193us) server_negotiation.cc:407] Connection header received
0810 18:45:42.293633 (+ 2055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:42.293639 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:42.293766 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:42.293898 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:42.296341 (+ 2443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:42.297384 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:42.299135 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:42.299520 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:42.299625 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:42.303360 (+ 3735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:42.303366 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:42.303369 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:42.303420 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:42.306028 (+ 2608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:42.306820 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:42.306824 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:42.306826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:42.306952 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:42.307507 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:42.307511 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:42.307514 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:42.307923 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:42.308111 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:42.309223 (+ 1112us) server_negotiation.cc:299] Negotiation successful
0810 18:45:42.309461 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":311,"threads_started":1}
I20260810 18:45:42.310956 29592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:42.290667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.195:37633)
0810 18:45:42.291399 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:42.291447 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:42.293358 (+ 1911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:42.294246 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:42.294259 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:42.295264 (+ 1005us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:45:42.296136 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:42.296159 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:42.297564 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:42.297569 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:42.298979 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:42.298993 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:42.299781 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:42.299788 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:42.299997 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:42.300739 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:45:42.300767 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:45:42.303125 (+ 2358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:45:42.306167 (+ 3042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:42.306175 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:42.306187 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:42.306687 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:42.307078 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:45:42.307083 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:45:42.307086 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:45:42.307354 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:45:42.308083 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:45:42.308092 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:45:42.308680 (+ 588us) client_negotiation.cc:769] Sending connection context
0810 18:45:42.309077 (+ 397us) client_negotiation.cc:240] Negotiation successful
0810 18:45:42.309638 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":288,"threads_started":1}
I20260810 18:45:42.312422 29587 heartbeater.cc:347] Connected to a master server at 127.4.228.254:35121
I20260810 18:45:42.312757 29587 heartbeater.cc:464] Registering TS with master...
I20260810 18:45:42.313525 29587 heartbeater.cc:511] Master 127.4.228.254:35121 requested a full tablet report, sending...
I20260810 18:45:42.315037 28902 ts_manager.cc:194] Registered new tserver with Master: 23521ea509214ca89745701374ff8eda (127.4.228.195:40527)
I20260810 18:45:42.315881 28902 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.195@KRBTEST.COM'} at 127.4.228.195:37633
I20260810 18:45:42.322293 5011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:45:42.340734 29591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:42.327174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49874 (local address 127.4.228.254:35121)
0810 18:45:42.327361 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:42.327367 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:42.327849 (+ 482us) server_negotiation.cc:407] Connection header received
0810 18:45:42.328004 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:42.328008 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:42.328080 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:42.328169 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:42.329681 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:42.330437 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:42.331861 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:42.332308 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:42.332417 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:42.335280 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:42.335285 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:42.335288 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:42.335326 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:42.338144 (+ 2818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:42.338868 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:42.338872 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:42.338875 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:42.339002 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:42.339393 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:42.339396 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:42.339398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:42.339737 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:42.339922 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:42.340307 (+ 385us) server_negotiation.cc:299] Negotiation successful
0810 18:45:42.340480 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 18:45:42.345177 28902 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49874:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:45:42.352599 28902 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:45:42.390616 29604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:42.366569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:39905 (local address 127.0.0.1:53466)
0810 18:45:42.380017 (+ 13448us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:42.380078 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:42.380396 (+ 318us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:42.382256 (+ 1860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:42.382264 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:42.382317 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:45:42.383279 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:42.383294 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:42.385303 (+ 2009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:42.385310 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:42.387758 (+ 2448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:42.387772 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:42.389447 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:42.389453 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:42.389657 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:42.389720 (+ 63us) client_negotiation.cc:769] Sending connection context
0810 18:45:42.389877 (+ 157us) client_negotiation.cc:240] Negotiation successful
0810 18:45:42.390081 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13075,"mutex_wait_us":194,"spinlock_wait_cycles":768,"thread_start_us":15410,"threads_started":1}
I20260810 18:45:42.390760 29603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:42.367107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:45331 (local address 127.0.0.1:38956)
0810 18:45:42.382257 (+ 15150us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:42.382282 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:42.382439 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:42.383008 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:42.383012 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:42.383041 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:45:42.383607 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:42.383631 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:42.385395 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:42.385402 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:42.388027 (+ 2625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:42.388043 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:42.389893 (+ 1850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:42.389899 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:42.390063 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:42.390091 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 18:45:42.390193 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 18:45:42.390327 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15024,"mutex_wait_us":1,"thread_start_us":13717,"threads_started":1}
I20260810 18:45:42.390784 29611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:42.368295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38956 (local address 127.4.228.194:45331)
0810 18:45:42.377563 (+ 9268us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:42.377575 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:42.382408 (+ 4833us) server_negotiation.cc:407] Connection header received
0810 18:45:42.382625 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:42.382632 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:42.382809 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:42.383011 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:45:42.383832 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:42.385193 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:42.388200 (+ 3007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:42.389698 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:42.389825 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:42.390026 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:42.390268 (+ 242us) server_negotiation.cc:299] Negotiation successful
0810 18:45:42.390455 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9097,"thread_start_us":8998,"threads_started":1}
I20260810 18:45:42.391515 29605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:42.366887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53466 (local address 127.4.228.193:39905)
0810 18:45:42.379227 (+ 12340us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:42.379240 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:42.381306 (+ 2066us) server_negotiation.cc:407] Connection header received
0810 18:45:42.381428 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:42.381436 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:42.381722 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:42.381928 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:45:42.383507 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:42.385149 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:42.387922 (+ 2773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:42.389248 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:42.389415 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:42.389702 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:42.391101 (+ 1399us) server_negotiation.cc:299] Negotiation successful
0810 18:45:42.391249 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12085,"thread_start_us":14167,"threads_started":1}
I20260810 18:45:42.392516 29607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:42.367651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45008 (local address 127.4.228.195:40527)
0810 18:45:42.377107 (+ 9456us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:42.377119 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:42.381418 (+ 4299us) server_negotiation.cc:407] Connection header received
0810 18:45:42.381516 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:42.381522 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:42.381707 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:42.381928 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:45:42.383886 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:42.385864 (+ 1978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:42.388958 (+ 3094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:42.390387 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:42.390533 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:42.391850 (+ 1317us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:42.391935 (+ 85us) server_negotiation.cc:299] Negotiation successful
0810 18:45:42.392120 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9230,"thread_start_us":9044,"threads_started":1}
I20260810 18:45:42.393355 29606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:42.366763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:40527 (local address 127.0.0.1:45008)
0810 18:45:42.381118 (+ 14355us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:42.381138 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:42.381280 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:42.382262 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:42.382266 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:42.382324 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:45:42.383279 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:42.383294 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:42.385998 (+ 2704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:42.386003 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:42.388761 (+ 2758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:42.388777 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:42.391321 (+ 2544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:42.391328 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:42.391485 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:42.391515 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 18:45:42.391591 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 18:45:42.393040 (+ 1449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14218,"mutex_wait_us":167,"spinlock_wait_cycles":9344,"thread_start_us":12719,"threads_started":1}
I20260810 18:45:42.395134 29302 tablet_service.cc:1467] Processing CreateTablet for tablet 731eb4951a2a4d75ae3dddf6e99c2850 (DEFAULT_TABLE table=test-table [id=d64c88434fb04351ab5b5af3923d74e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:45:42.395138 29085 tablet_service.cc:1467] Processing CreateTablet for tablet 731eb4951a2a4d75ae3dddf6e99c2850 (DEFAULT_TABLE table=test-table [id=d64c88434fb04351ab5b5af3923d74e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:45:42.396739 29302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 731eb4951a2a4d75ae3dddf6e99c2850. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:45:42.396737 29085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 731eb4951a2a4d75ae3dddf6e99c2850. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:45:42.396409 29493 tablet_service.cc:1467] Processing CreateTablet for tablet 731eb4951a2a4d75ae3dddf6e99c2850 (DEFAULT_TABLE table=test-table [id=d64c88434fb04351ab5b5af3923d74e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:45:42.397706 29493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 731eb4951a2a4d75ae3dddf6e99c2850. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:45:42.418823 29614 tablet_bootstrap.cc:492] T 731eb4951a2a4d75ae3dddf6e99c2850 P c04280931825462796891c073e8074ac: Bootstrap starting.
I20260810 18:45:42.422300 29616 tablet_bootstrap.cc:492] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864: Bootstrap starting.
I20260810 18:45:42.425971 29616 tablet_bootstrap.cc:654] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864: Neither blocks nor log segments found. Creating new log.
I20260810 18:45:42.427176 29616 log.cc:824] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864: Log is configured to *not* fsync() on all Append() calls
I20260810 18:45:42.430052 29616 tablet_bootstrap.cc:492] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864: No bootstrap required, opened a new log
I20260810 18:45:42.430399 29616 ts_tablet_manager.cc:1397] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260810 18:45:42.431073 29617 tablet_bootstrap.cc:492] T 731eb4951a2a4d75ae3dddf6e99c2850 P 23521ea509214ca89745701374ff8eda: Bootstrap starting.
I20260810 18:45:42.432045 29614 tablet_bootstrap.cc:654] T 731eb4951a2a4d75ae3dddf6e99c2850 P c04280931825462796891c073e8074ac: Neither blocks nor log segments found. Creating new log.
I20260810 18:45:42.433666 29614 log.cc:824] T 731eb4951a2a4d75ae3dddf6e99c2850 P c04280931825462796891c073e8074ac: Log is configured to *not* fsync() on all Append() calls
I20260810 18:45:42.435177 29617 tablet_bootstrap.cc:654] T 731eb4951a2a4d75ae3dddf6e99c2850 P 23521ea509214ca89745701374ff8eda: Neither blocks nor log segments found. Creating new log.
I20260810 18:45:42.436470 29617 log.cc:824] T 731eb4951a2a4d75ae3dddf6e99c2850 P 23521ea509214ca89745701374ff8eda: Log is configured to *not* fsync() on all Append() calls
I20260810 18:45:42.436966 29614 tablet_bootstrap.cc:492] T 731eb4951a2a4d75ae3dddf6e99c2850 P c04280931825462796891c073e8074ac: No bootstrap required, opened a new log
I20260810 18:45:42.437363 29614 ts_tablet_manager.cc:1397] T 731eb4951a2a4d75ae3dddf6e99c2850 P c04280931825462796891c073e8074ac: Time spent bootstrapping tablet: real 0.019s user 0.009s sys 0.000s
I20260810 18:45:42.439787 29617 tablet_bootstrap.cc:492] T 731eb4951a2a4d75ae3dddf6e99c2850 P 23521ea509214ca89745701374ff8eda: No bootstrap required, opened a new log
I20260810 18:45:42.440057 29617 ts_tablet_manager.cc:1397] T 731eb4951a2a4d75ae3dddf6e99c2850 P 23521ea509214ca89745701374ff8eda: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260810 18:45:42.441489 29616 raft_consensus.cc:348] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3864ce298e8744258f3cdc6c68f97864" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 39905 } } peers { permanent_uuid: "23521ea509214ca89745701374ff8eda" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40527 } } peers { permanent_uuid: "c04280931825462796891c073e8074ac" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 45331 } }
I20260810 18:45:42.441946 29616 raft_consensus.cc:374] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:45:42.442093 29616 raft_consensus.cc:729] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3864ce298e8744258f3cdc6c68f97864, State: Initialized, Role: FOLLOWER
I20260810 18:45:42.442796 29616 consensus_queue.cc:260] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3864ce298e8744258f3cdc6c68f97864" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 39905 } } peers { permanent_uuid: "23521ea509214ca89745701374ff8eda" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40527 } } peers { permanent_uuid: "c04280931825462796891c073e8074ac" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 45331 } }
I20260810 18:45:42.444656 29616 ts_tablet_manager.cc:1428] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864: Time spent starting tablet: real 0.014s user 0.011s sys 0.002s
I20260810 18:45:42.445744 29614 raft_consensus.cc:348] T 731eb4951a2a4d75ae3dddf6e99c2850 P c04280931825462796891c073e8074ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3864ce298e8744258f3cdc6c68f97864" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 39905 } } peers { permanent_uuid: "23521ea509214ca89745701374ff8eda" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40527 } } peers { permanent_uuid: "c04280931825462796891c073e8074ac" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 45331 } }
I20260810 18:45:42.446131 29614 raft_consensus.cc:374] T 731eb4951a2a4d75ae3dddf6e99c2850 P c04280931825462796891c073e8074ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:45:42.446256 29614 raft_consensus.cc:729] T 731eb4951a2a4d75ae3dddf6e99c2850 P c04280931825462796891c073e8074ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c04280931825462796891c073e8074ac, State: Initialized, Role: FOLLOWER
I20260810 18:45:42.446991 29614 consensus_queue.cc:260] T 731eb4951a2a4d75ae3dddf6e99c2850 P c04280931825462796891c073e8074ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3864ce298e8744258f3cdc6c68f97864" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 39905 } } peers { permanent_uuid: "23521ea509214ca89745701374ff8eda" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40527 } } peers { permanent_uuid: "c04280931825462796891c073e8074ac" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 45331 } }
I20260810 18:45:42.448159 29396 heartbeater.cc:503] Master 127.4.228.254:35121 was elected leader, sending a full tablet report...
I20260810 18:45:42.448916 29614 ts_tablet_manager.cc:1428] T 731eb4951a2a4d75ae3dddf6e99c2850 P c04280931825462796891c073e8074ac: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
W20260810 18:45:42.450708 29399 tablet.cc:2369] T 731eb4951a2a4d75ae3dddf6e99c2850 P c04280931825462796891c073e8074ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:45:42.451526 29617 raft_consensus.cc:348] T 731eb4951a2a4d75ae3dddf6e99c2850 P 23521ea509214ca89745701374ff8eda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3864ce298e8744258f3cdc6c68f97864" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 39905 } } peers { permanent_uuid: "23521ea509214ca89745701374ff8eda" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40527 } } peers { permanent_uuid: "c04280931825462796891c073e8074ac" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 45331 } }
I20260810 18:45:42.452039 29617 raft_consensus.cc:374] T 731eb4951a2a4d75ae3dddf6e99c2850 P 23521ea509214ca89745701374ff8eda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:45:42.452167 29617 raft_consensus.cc:729] T 731eb4951a2a4d75ae3dddf6e99c2850 P 23521ea509214ca89745701374ff8eda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23521ea509214ca89745701374ff8eda, State: Initialized, Role: FOLLOWER
I20260810 18:45:42.452850 29617 consensus_queue.cc:260] T 731eb4951a2a4d75ae3dddf6e99c2850 P 23521ea509214ca89745701374ff8eda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3864ce298e8744258f3cdc6c68f97864" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 39905 } } peers { permanent_uuid: "23521ea509214ca89745701374ff8eda" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40527 } } peers { permanent_uuid: "c04280931825462796891c073e8074ac" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 45331 } }
I20260810 18:45:42.453987 29587 heartbeater.cc:503] Master 127.4.228.254:35121 was elected leader, sending a full tablet report...
I20260810 18:45:42.454712 29617 ts_tablet_manager.cc:1428] T 731eb4951a2a4d75ae3dddf6e99c2850 P 23521ea509214ca89745701374ff8eda: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260810 18:45:42.507568 29621 raft_consensus.cc:482] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:45:42.507990 29621 raft_consensus.cc:504] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3864ce298e8744258f3cdc6c68f97864" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 39905 } } peers { permanent_uuid: "23521ea509214ca89745701374ff8eda" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40527 } } peers { permanent_uuid: "c04280931825462796891c073e8074ac" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 45331 } }
I20260810 18:45:42.510094 29621 leader_election.cc:288] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23521ea509214ca89745701374ff8eda (127.4.228.195:40527), c04280931825462796891c073e8074ac (127.4.228.194:45331)
I20260810 18:45:42.516634 29607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:42.510392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:40661 (local address 127.4.228.195:40527)
0810 18:45:42.510655 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:42.510665 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:42.511400 (+ 735us) server_negotiation.cc:407] Connection header received
0810 18:45:42.511698 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:42.511702 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:42.511763 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:42.511862 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:45:42.512399 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:42.513111 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:42.514914 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:42.515682 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:42.515885 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:42.515965 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:42.516239 (+ 274us) server_negotiation.cc:299] Negotiation successful
0810 18:45:42.516368 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 18:45:42.516908 29626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:42.510328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:40527 (local address 127.4.228.193:40661)
0810 18:45:42.511274 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:42.511296 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:42.511486 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:42.511883 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:42.511888 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:42.511934 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:45:42.512255 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:42.512264 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:42.513250 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:42.513257 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:42.514758 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:42.514771 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:42.515897 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:42.515903 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:42.516079 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:42.516103 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 18:45:42.516211 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 18:45:42.516350 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":832,"spinlock_wait_cycles":1408,"thread_start_us":595,"threads_started":1}
I20260810 18:45:42.517570 29627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:42.510651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:45331 (local address 127.4.228.193:41647)
0810 18:45:42.511716 (+ 1065us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:42.511738 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 18:45:42.511916 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:45:42.512305 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:45:42.512310 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:45:42.512335 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:45:42.512724 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:42.512734 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:42.513810 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:42.513817 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:42.515332 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:45:42.515345 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:45:42.516620 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:45:42.516627 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:45:42.516882 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:42.516921 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 18:45:42.517107 (+ 186us) client_negotiation.cc:240] Negotiation successful
0810 18:45:42.517274 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":946,"thread_start_us":622,"threads_started":1}
I20260810 18:45:42.517588 29611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:42.510755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:41647 (local address 127.4.228.194:45331)
0810 18:45:42.511012 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:42.511037 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:42.511857 (+ 820us) server_negotiation.cc:407] Connection header received
0810 18:45:42.512108 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:42.512114 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:42.512182 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:42.512291 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:45:42.512907 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:42.513650 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:42.515524 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:42.516357 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:42.516557 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:42.516651 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:42.517173 (+ 522us) server_negotiation.cc:299] Negotiation successful
0810 18:45:42.517324 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 18:45:42.518148 29518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "731eb4951a2a4d75ae3dddf6e99c2850" candidate_uuid: "3864ce298e8744258f3cdc6c68f97864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23521ea509214ca89745701374ff8eda" is_pre_election: true
I20260810 18:45:42.518601 29333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "731eb4951a2a4d75ae3dddf6e99c2850" candidate_uuid: "3864ce298e8744258f3cdc6c68f97864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c04280931825462796891c073e8074ac" is_pre_election: true
I20260810 18:45:42.518751 29518 raft_consensus.cc:2454] T 731eb4951a2a4d75ae3dddf6e99c2850 P 23521ea509214ca89745701374ff8eda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3864ce298e8744258f3cdc6c68f97864 in term 0.
I20260810 18:45:42.519107 29333 raft_consensus.cc:2454] T 731eb4951a2a4d75ae3dddf6e99c2850 P c04280931825462796891c073e8074ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3864ce298e8744258f3cdc6c68f97864 in term 0.
I20260810 18:45:42.519567 29021 leader_election.cc:302] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23521ea509214ca89745701374ff8eda, 3864ce298e8744258f3cdc6c68f97864; no voters:
I20260810 18:45:42.519996 29621 raft_consensus.cc:2781] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:45:42.520180 29621 raft_consensus.cc:482] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:45:42.520325 29621 raft_consensus.cc:3037] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:45:42.522720 29621 raft_consensus.cc:504] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3864ce298e8744258f3cdc6c68f97864" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 39905 } } peers { permanent_uuid: "23521ea509214ca89745701374ff8eda" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40527 } } peers { permanent_uuid: "c04280931825462796891c073e8074ac" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 45331 } }
I20260810 18:45:42.523435 29621 leader_election.cc:288] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864 [CANDIDATE]: Term 1 election: Requested vote from peers 23521ea509214ca89745701374ff8eda (127.4.228.195:40527), c04280931825462796891c073e8074ac (127.4.228.194:45331)
I20260810 18:45:42.523979 29518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "731eb4951a2a4d75ae3dddf6e99c2850" candidate_uuid: "3864ce298e8744258f3cdc6c68f97864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23521ea509214ca89745701374ff8eda"
I20260810 18:45:42.524032 29333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "731eb4951a2a4d75ae3dddf6e99c2850" candidate_uuid: "3864ce298e8744258f3cdc6c68f97864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c04280931825462796891c073e8074ac"
I20260810 18:45:42.524216 29518 raft_consensus.cc:3037] T 731eb4951a2a4d75ae3dddf6e99c2850 P 23521ea509214ca89745701374ff8eda [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:45:42.524339 29333 raft_consensus.cc:3037] T 731eb4951a2a4d75ae3dddf6e99c2850 P c04280931825462796891c073e8074ac [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:45:42.527012 29518 raft_consensus.cc:2454] T 731eb4951a2a4d75ae3dddf6e99c2850 P 23521ea509214ca89745701374ff8eda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3864ce298e8744258f3cdc6c68f97864 in term 1.
I20260810 18:45:42.527349 29333 raft_consensus.cc:2454] T 731eb4951a2a4d75ae3dddf6e99c2850 P c04280931825462796891c073e8074ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3864ce298e8744258f3cdc6c68f97864 in term 1.
I20260810 18:45:42.527571 29021 leader_election.cc:302] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23521ea509214ca89745701374ff8eda, 3864ce298e8744258f3cdc6c68f97864; no voters:
I20260810 18:45:42.527962 29621 raft_consensus.cc:2781] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:45:42.528475 29621 raft_consensus.cc:686] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864 [term 1 LEADER]: Becoming Leader. State: Replica: 3864ce298e8744258f3cdc6c68f97864, State: Running, Role: LEADER
I20260810 18:45:42.529112 29621 consensus_queue.cc:237] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3864ce298e8744258f3cdc6c68f97864" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 39905 } } peers { permanent_uuid: "23521ea509214ca89745701374ff8eda" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40527 } } peers { permanent_uuid: "c04280931825462796891c073e8074ac" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 45331 } }
I20260810 18:45:42.534255 28901 catalog_manager.cc:5495] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3864ce298e8744258f3cdc6c68f97864 (127.4.228.193). New cstate: current_term: 1 leader_uuid: "3864ce298e8744258f3cdc6c68f97864" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3864ce298e8744258f3cdc6c68f97864" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 39905 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23521ea509214ca89745701374ff8eda" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c04280931825462796891c073e8074ac" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 45331 } health_report { overall_health: UNKNOWN } } }
W20260810 18:45:42.538805 29590 tablet.cc:2369] T 731eb4951a2a4d75ae3dddf6e99c2850 P 23521ea509214ca89745701374ff8eda: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:45:42.587380 29605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:42.580778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53476 (local address 127.4.228.193:39905)
0810 18:45:42.581158 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:42.581166 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:42.581256 (+ 90us) server_negotiation.cc:407] Connection header received
0810 18:45:42.581551 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:42.581555 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:42.581667 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:42.581846 (+ 179us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:45:42.582299 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:42.583207 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:42.584791 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:42.585265 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:42.585421 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:42.586006 (+ 585us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:45:42.586625 (+ 619us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:45:42.586813 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:42.586930 (+ 117us) server_negotiation.cc:299] Negotiation successful
0810 18:45:42.587051 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175}
I20260810 18:45:42.597399 29518 raft_consensus.cc:1261] T 731eb4951a2a4d75ae3dddf6e99c2850 P 23521ea509214ca89745701374ff8eda [term 1 FOLLOWER]: Refusing update from remote peer 3864ce298e8744258f3cdc6c68f97864: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:45:42.597397 29333 raft_consensus.cc:1261] T 731eb4951a2a4d75ae3dddf6e99c2850 P c04280931825462796891c073e8074ac [term 1 FOLLOWER]: Refusing update from remote peer 3864ce298e8744258f3cdc6c68f97864: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:45:42.598591 29628 consensus_queue.cc:1035] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23521ea509214ca89745701374ff8eda" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:45:42.599035 29621 consensus_queue.cc:1035] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c04280931825462796891c073e8074ac" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:45:42.611173 29633 mvcc.cc:205] Tried to move back new op lower bound from 7317043374467530752 to 7317043374203760640. Current Snapshot: MvccSnapshot[applied={T|T < 7317043374467530752}]
I20260810 18:45:42.612598 29631 mvcc.cc:205] Tried to move back new op lower bound from 7317043374467530752 to 7317043374203760640. Current Snapshot: MvccSnapshot[applied={T|T < 7317043374467530752 or (T in {7317043374467530752})}]
I20260810 18:45:42.626065 28901 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:45:42.626454 28901 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:45:42.631584 28901 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca: Sending DeleteTablet for 3 replicas of tablet 731eb4951a2a4d75ae3dddf6e99c2850
I20260810 18:45:42.632907 29302 tablet_service.cc:1514] Processing DeleteTablet for tablet 731eb4951a2a4d75ae3dddf6e99c2850 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:45:42 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:38956
I20260810 18:45:42.633103 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 28994
I20260810 18:45:42.633069 29492 tablet_service.cc:1514] Processing DeleteTablet for tablet 731eb4951a2a4d75ae3dddf6e99c2850 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:45:42 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:45008
I20260810 18:45:42.633347 29493 tablet_service.cc:1514] Processing DeleteTablet for tablet 731eb4951a2a4d75ae3dddf6e99c2850 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:45:42 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:45008
I20260810 18:45:42.634349 29085 tablet_service.cc:1514] Processing DeleteTablet for tablet 731eb4951a2a4d75ae3dddf6e99c2850 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:45:42 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:53466
I20260810 18:45:42.650017 29645 tablet_replica.cc:330] T 731eb4951a2a4d75ae3dddf6e99c2850 P c04280931825462796891c073e8074ac: stopping tablet replica
I20260810 18:45:42.651322 29646 tablet_replica.cc:330] T 731eb4951a2a4d75ae3dddf6e99c2850 P 23521ea509214ca89745701374ff8eda: stopping tablet replica
I20260810 18:45:42.651913 29646 raft_consensus.cc:2229] T 731eb4951a2a4d75ae3dddf6e99c2850 P 23521ea509214ca89745701374ff8eda [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:45:42.651985 29645 raft_consensus.cc:2229] T 731eb4951a2a4d75ae3dddf6e99c2850 P c04280931825462796891c073e8074ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:45:42.652601 29646 raft_consensus.cc:2258] T 731eb4951a2a4d75ae3dddf6e99c2850 P 23521ea509214ca89745701374ff8eda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:45:42.655117 29646 ts_tablet_manager.cc:1905] T 731eb4951a2a4d75ae3dddf6e99c2850 P 23521ea509214ca89745701374ff8eda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:45:42.655396 29648 tablet_replica.cc:330] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864: stopping tablet replica
I20260810 18:45:42.655768 29648 raft_consensus.cc:2229] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:45:42.656807 29645 raft_consensus.cc:2258] T 731eb4951a2a4d75ae3dddf6e99c2850 P c04280931825462796891c073e8074ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:45:42.657375 29648 raft_consensus.cc:2258] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:45:42.659410 29645 ts_tablet_manager.cc:1905] T 731eb4951a2a4d75ae3dddf6e99c2850 P c04280931825462796891c073e8074ac: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 10 18:45:42 dist-test-slave-7kzw krb5kdc[28843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.193@KRBTEST.COM
I20260810 18:45:42.662680 29646 ts_tablet_manager.cc:1918] T 731eb4951a2a4d75ae3dddf6e99c2850 P 23521ea509214ca89745701374ff8eda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:45:42.662879 29646 log.cc:1196] T 731eb4951a2a4d75ae3dddf6e99c2850 P 23521ea509214ca89745701374ff8eda: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/wal/wals/731eb4951a2a4d75ae3dddf6e99c2850
I20260810 18:45:42.663520 29646 ts_tablet_manager.cc:1939] T 731eb4951a2a4d75ae3dddf6e99c2850 P 23521ea509214ca89745701374ff8eda: Deleting consensus metadata
I20260810 18:45:42.664448 29648 ts_tablet_manager.cc:1905] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:45:42.665930 28884 catalog_manager.cc:4841] TS 23521ea509214ca89745701374ff8eda (127.4.228.195:40527): tablet 731eb4951a2a4d75ae3dddf6e99c2850 (table test-table [id=d64c88434fb04351ab5b5af3923d74e0]) successfully deleted
W20260810 18:45:42.667129 28884 catalog_manager.cc:4808] TS 23521ea509214ca89745701374ff8eda (127.4.228.195:40527): delete failed for tablet 731eb4951a2a4d75ae3dddf6e99c2850 because the tablet was not found. No further retry: Not found: Tablet not found: 731eb4951a2a4d75ae3dddf6e99c2850
I20260810 18:45:42.668265 29645 ts_tablet_manager.cc:1918] T 731eb4951a2a4d75ae3dddf6e99c2850 P c04280931825462796891c073e8074ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:45:42.668457 29645 log.cc:1196] T 731eb4951a2a4d75ae3dddf6e99c2850 P c04280931825462796891c073e8074ac: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/wal/wals/731eb4951a2a4d75ae3dddf6e99c2850
I20260810 18:45:42.669035 29645 ts_tablet_manager.cc:1939] T 731eb4951a2a4d75ae3dddf6e99c2850 P c04280931825462796891c073e8074ac: Deleting consensus metadata
I20260810 18:45:42.671075 28883 catalog_manager.cc:4841] TS c04280931825462796891c073e8074ac (127.4.228.194:45331): tablet 731eb4951a2a4d75ae3dddf6e99c2850 (table test-table [id=d64c88434fb04351ab5b5af3923d74e0]) successfully deleted
I20260810 18:45:42.672214 29605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:42.634922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53492 (local address 127.4.228.193:39905)
0810 18:45:42.635183 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:42.635192 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:42.651164 (+ 15972us) server_negotiation.cc:407] Connection header received
0810 18:45:42.651465 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:42.651472 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:42.651564 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:42.651687 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:42.653954 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:42.655237 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:42.657417 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:42.658231 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:42.658397 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:42.664238 (+ 5841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:42.664265 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:42.664282 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:42.664337 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:42.668385 (+ 4048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:42.669252 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:42.669262 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:42.669266 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:42.669430 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:42.669875 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:42.669881 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:42.669888 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:42.670265 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:42.670640 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:42.671755 (+ 1115us) server_negotiation.cc:299] Negotiation successful
0810 18:45:42.671945 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 18:45:42.672955 29185 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:45:42.772789 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 28994
W20260810 18:45:42.803785 28885 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.228.193:39905 (error 108)
I20260810 18:45:42.804668 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 29207
I20260810 18:45:42.806165 29606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:42.805494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:39905 (local address 127.0.0.1:53494)
0810 18:45:42.805841 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:42.805975 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.193:39905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207}
W20260810 18:45:42.806445 28885 catalog_manager.cc:4584] TS 3864ce298e8744258f3cdc6c68f97864 (127.4.228.193:39905): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 731eb4951a2a4d75ae3dddf6e99c2850: Network error: Client connection negotiation failed: client connection to 127.4.228.193:39905: connect: Connection refused (error 111)
Aug 10 18:45:42 dist-test-slave-7kzw krb5kdc[28843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.194@KRBTEST.COM
I20260810 18:45:42.822101 29611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:42.805764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38960 (local address 127.4.228.194:45331)
0810 18:45:42.805981 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:42.805987 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:42.806011 (+ 24us) server_negotiation.cc:407] Connection header received
0810 18:45:42.806083 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:42.806089 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:42.806192 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:42.806318 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:42.808097 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:42.809075 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:42.810711 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:42.811127 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:42.811257 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:42.815754 (+ 4497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:42.815771 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:42.815829 (+ 58us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:42.815890 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:42.819019 (+ 3129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:42.819820 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:42.819829 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:42.819831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:42.819981 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:42.820418 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:42.820423 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:42.820425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:42.820863 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:42.821117 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:42.821518 (+ 401us) server_negotiation.cc:299] Negotiation successful
0810 18:45:42.821788 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260810 18:45:42.822958 29391 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:45:42.860319 29606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:42.859750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:39905 (local address 127.0.0.1:53496)
0810 18:45:42.860003 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:42.860116 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.193:39905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260810 18:45:42.861510 29606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:42.861113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:39905 (local address 127.0.0.1:53508)
0810 18:45:42.861300 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:42.861376 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.193:39905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 18:45:42.914552 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 29207
I20260810 18:45:42.922041 29606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:42.921562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:39905 (local address 127.0.0.1:53510)
0810 18:45:42.921792 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:42.921882 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.193:39905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260810 18:45:42.923460 29606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:42.923048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:39905 (local address 127.0.0.1:53526)
0810 18:45:42.923227 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:42.923317 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.193:39905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260810 18:45:42.943327 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 29408
Aug 10 18:45:42 dist-test-slave-7kzw krb5kdc[28843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.195@KRBTEST.COM
I20260810 18:45:42.960644 29607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:42.944259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45020 (local address 127.4.228.195:40527)
0810 18:45:42.944457 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 18:45:42.944464 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:45:42.944556 (+ 92us) server_negotiation.cc:407] Connection header received
0810 18:45:42.944774 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:45:42.944782 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:45:42.944871 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:45:42.945005 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:45:42.946367 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:42.947217 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:42.948815 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:45:42.949261 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:45:42.949402 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:45:42.954293 (+ 4891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:45:42.954308 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:45:42.954328 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:45:42.954364 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:45:42.957592 (+ 3228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:42.958366 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:42.958375 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:42.958378 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:42.958671 (+ 293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:45:42.959063 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:45:42.959066 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:45:42.959068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:45:42.959487 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:45:42.959742 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0810 18:45:42.960031 (+ 289us) server_negotiation.cc:299] Negotiation successful
0810 18:45:42.960233 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 18:45:42.961362 29581 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:45:43.005905 29606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:43.005249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:39905 (local address 127.0.0.1:53538)
0810 18:45:43.005545 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:43.005663 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.193:39905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260810 18:45:43.007360 29606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:43.006848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:39905 (local address 127.0.0.1:53540)
0810 18:45:43.007101 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:43.007197 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.193:39905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
I20260810 18:45:43.075488 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 29408
I20260810 18:45:43.104166 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 28862
I20260810 18:45:43.105562 28945 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:45:43.218549 29606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:45:43.217822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:39905 (local address 127.0.0.1:53542)
0810 18:45:43.218083 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0810 18:45:43.218227 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.193:39905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260810 18:45:43.218935 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 28862
I20260810 18:46:03.251886 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:35121
--webserver_interface=127.4.228.254
--webserver_port=34223
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:34661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:35121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:46:03.606957 29658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:46:03.607357 29658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:46:03.607532 29658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:46:03.615460 29658 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 18:46:03.615581 29658 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 18:46:03.615649 29658 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 18:46:03.615736 29658 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:46:03.615834 29658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:46:03.615931 29658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:46:03.615996 29658 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:46:03.616075 29658 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:46:03.626355 29658 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:34661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:35121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:35121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=34223
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:46:03.627810 29658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:46:03.629210 29658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:46:03.655475 29666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:03.655622 29669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:03.655593 29671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:46:03.656279 29658 server_base.cc:1034] running on GCE node
I20260810 18:46:03.657235 29658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:46:03.659001 29658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:46:03.660295 29658 hybrid_clock.cc:648] HybridClock initialized: now 1786387563660206 us; error 55 us; skew 500 ppm
I20260810 18:46:03.660727 29658 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:46:03 dist-test-slave-7kzw krb5kdc[28843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387563, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:46:03.666409 29658 init.cc:377] Logged in from keytab as kudu/127.4.228.254@KRBTEST.COM (short username kudu)
I20260810 18:46:03.669070 29658 webserver.cc:460] Webserver started at http://127.4.228.254:34223/ using document root <none> and password file <none>
I20260810 18:46:03.670301 29658 fs_manager.cc:359] Metadata directory not provided
I20260810 18:46:03.670518 29658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:46:03.678038 29658 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 18:46:03.681429 29681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:03.683164 29658 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 18:46:03.683405 29658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "4d47ce6d53164cc6b711addede1613ca"
format_stamp: "Formatted at 2026-08-10 18:45:40 on dist-test-slave-7kzw"
server_key: "e80ea1563709bcc7d4ff985586949483"
server_key_iv: "47a9842fb3eff52a620674c42de1c8fe"
server_key_version: "encryptionkey@0"
I20260810 18:46:03.684592 29658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:46:03.711170 29658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:46:03.716259 29658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:46:03.716658 29658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:46:03.738444 29658 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:35121
I20260810 18:46:03.738515 29770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:35121 every 8 connection(s)
I20260810 18:46:03.740170 29658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:46:03.744167 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 29658
I20260810 18:46:03.745234 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.193:39905
--local_ip_for_outbound_sockets=127.4.228.193
--tserver_master_addrs=127.4.228.254:35121
--webserver_port=43447
--webserver_interface=127.4.228.193
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:34661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:46:03.750272 29772 sys_catalog.cc:263] Verifying existing consensus state
I20260810 18:46:03.755990 29772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca: Bootstrap starting.
I20260810 18:46:03.777773 29772 log.cc:824] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca: Log is configured to *not* fsync() on all Append() calls
I20260810 18:46:03.791754 29772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:46:03.792361 29772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca: Bootstrap complete.
I20260810 18:46:03.802132 29772 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d47ce6d53164cc6b711addede1613ca" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35121 } }
I20260810 18:46:03.803097 29772 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d47ce6d53164cc6b711addede1613ca, State: Initialized, Role: FOLLOWER
I20260810 18:46:03.803862 29772 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d47ce6d53164cc6b711addede1613ca" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35121 } }
I20260810 18:46:03.804173 29772 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:46:03.804311 29772 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:46:03.804522 29772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [term 1 FOLLOWER]: Advancing to term 2
I20260810 18:46:03.808665 29772 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d47ce6d53164cc6b711addede1613ca" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35121 } }
I20260810 18:46:03.809255 29772 leader_election.cc:302] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d47ce6d53164cc6b711addede1613ca; no voters:
I20260810 18:46:03.809985 29772 leader_election.cc:288] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 18:46:03.810135 29778 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [term 2 FOLLOWER]: Leader election won for term 2
I20260810 18:46:03.811450 29778 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [term 2 LEADER]: Becoming Leader. State: Replica: 4d47ce6d53164cc6b711addede1613ca, State: Running, Role: LEADER
I20260810 18:46:03.812111 29778 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d47ce6d53164cc6b711addede1613ca" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35121 } }
I20260810 18:46:03.812532 29772 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:46:03.814924 29779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4d47ce6d53164cc6b711addede1613ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d47ce6d53164cc6b711addede1613ca" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35121 } } }
I20260810 18:46:03.815410 29779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [sys.catalog]: This master's current role is: LEADER
I20260810 18:46:03.816180 29780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d47ce6d53164cc6b711addede1613ca. Latest consensus state: current_term: 2 leader_uuid: "4d47ce6d53164cc6b711addede1613ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d47ce6d53164cc6b711addede1613ca" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35121 } } }
I20260810 18:46:03.816509 29780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [sys.catalog]: This master's current role is: LEADER
I20260810 18:46:03.818720 29789 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:46:03.827872 29789 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:46:03.829373 29789 catalog_manager.cc:1259] Loaded cluster ID: f39befd62fef402e98e6aef83721c785
I20260810 18:46:03.829545 29789 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:46:03.833822 29789 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:46:03.834700 29804 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 18:46:03.838495 29789 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca: Loaded TSK: 0
I20260810 18:46:03.870118 29789 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca: Generated new TSK 1
I20260810 18:46:03.870604 29789 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 18:46:04.125088 29774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:46:04.125428 29774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:46:04.125589 29774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:46:04.132907 29774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:46:04.133068 29774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:46:04.133245 29774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.193
I20260810 18:46:04.143050 29774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:34661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.193:39905
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.228.193
--webserver_port=43447
--tserver_master_addrs=127.4.228.254:35121
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.193
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:46:04.144382 29774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:46:04.145823 29774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:46:04.162331 29816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:04.162566 29813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:04.162740 29812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:46:04.163673 29774 server_base.cc:1034] running on GCE node
I20260810 18:46:04.164381 29774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:46:04.165387 29774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:46:04.166648 29774 hybrid_clock.cc:648] HybridClock initialized: now 1786387564166589 us; error 38 us; skew 500 ppm
I20260810 18:46:04.167066 29774 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:46:04 dist-test-slave-7kzw krb5kdc[28843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387564, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:46:04.170878 29774 init.cc:377] Logged in from keytab as kudu/127.4.228.193@KRBTEST.COM (short username kudu)
I20260810 18:46:04.172525 29774 webserver.cc:460] Webserver started at http://127.4.228.193:43447/ using document root <none> and password file <none>
I20260810 18:46:04.173287 29774 fs_manager.cc:359] Metadata directory not provided
I20260810 18:46:04.173449 29774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:46:04.179726 29774 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260810 18:46:04.183681 29826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:04.185264 29774 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:46:04.185459 29774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/wal
uuid: "3864ce298e8744258f3cdc6c68f97864"
format_stamp: "Formatted at 2026-08-10 18:45:41 on dist-test-slave-7kzw"
server_key: "76cb203db86fc72e217f15ef5039d2a9"
server_key_iv: "d84ca856c058ce9d594105efeb53255e"
server_key_version: "encryptionkey@0"
I20260810 18:46:04.186451 29774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:46:04.242453 29774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:46:04.247943 29774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:46:04.248282 29774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:46:04.249470 29774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:46:04.251856 29837 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260810 18:46:04.258966 29774 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 18:46:04.259202 29774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.008s user 0.000s sys 0.001s
I20260810 18:46:04.259387 29774 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 18:46:04.263278 29774 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 18:46:04.263412 29774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.003s sys 0.001s
I20260810 18:46:04.263818 29837 tablet_bootstrap.cc:492] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864: Bootstrap starting.
I20260810 18:46:04.272867 29837 log.cc:824] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864: Log is configured to *not* fsync() on all Append() calls
I20260810 18:46:04.285808 29837 tablet_bootstrap.cc:492] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:46:04.286528 29837 tablet_bootstrap.cc:492] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864: Bootstrap complete.
I20260810 18:46:04.287248 29837 ts_tablet_manager.cc:1397] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864: Time spent bootstrapping tablet: real 0.024s user 0.017s sys 0.005s
I20260810 18:46:04.298344 29837 raft_consensus.cc:348] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3864ce298e8744258f3cdc6c68f97864" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 39905 } } peers { permanent_uuid: "23521ea509214ca89745701374ff8eda" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40527 } } peers { permanent_uuid: "c04280931825462796891c073e8074ac" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 45331 } }
I20260810 18:46:04.300276 29837 raft_consensus.cc:729] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3864ce298e8744258f3cdc6c68f97864, State: Initialized, Role: FOLLOWER
I20260810 18:46:04.301138 29837 consensus_queue.cc:260] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3864ce298e8744258f3cdc6c68f97864" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 39905 } } peers { permanent_uuid: "23521ea509214ca89745701374ff8eda" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40527 } } peers { permanent_uuid: "c04280931825462796891c073e8074ac" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 45331 } }
I20260810 18:46:04.303051 29837 ts_tablet_manager.cc:1428] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864: Time spent starting tablet: real 0.016s user 0.011s sys 0.005s
I20260810 18:46:04.317618 29774 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.193:39905
I20260810 18:46:04.317659 29993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.193:39905 every 8 connection(s)
I20260810 18:46:04.319483 29774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
I20260810 18:46:04.322247 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 29774
I20260810 18:46:04.323094 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.194:45331
--local_ip_for_outbound_sockets=127.4.228.194
--tserver_master_addrs=127.4.228.254:35121
--webserver_port=37951
--webserver_interface=127.4.228.194
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:34661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:46:04 dist-test-slave-7kzw krb5kdc[28843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387564, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:46:04.346866 29998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:04.323500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.193:47027)
0810 18:46:04.324638 (+ 1138us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:04.324693 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:04.325195 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:04.328200 (+ 3005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:04.328211 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:04.328936 (+ 725us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:46:04.329547 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:04.329566 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:04.331241 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:04.331247 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:04.332561 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:04.332580 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:04.333596 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:04.333603 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:04.333890 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:04.334661 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:46:04.334685 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:46:04.337335 (+ 2650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:46:04.342208 (+ 4873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:46:04.342216 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:46:04.342228 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:46:04.342833 (+ 605us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:46:04.343428 (+ 595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:46:04.343431 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:46:04.343434 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:46:04.343640 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:46:04.344686 (+ 1046us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:46:04.344692 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:46:04.344984 (+ 292us) client_negotiation.cc:769] Sending connection context
0810 18:46:04.345255 (+ 271us) client_negotiation.cc:240] Negotiation successful
0810 18:46:04.345649 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":754,"thread_start_us":426,"threads_started":1}
I20260810 18:46:04.347734 29997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:04.323548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:47027 (local address 127.4.228.254:35121)
0810 18:46:04.327018 (+ 3470us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:04.327040 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:04.327079 (+ 39us) server_negotiation.cc:407] Connection header received
0810 18:46:04.327378 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:04.327392 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:04.327708 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:04.328114 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:04.329777 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:04.331060 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:04.332727 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:04.333379 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:04.333590 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:04.337615 (+ 4025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:04.337627 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:04.337649 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:46:04.337705 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:04.342015 (+ 4310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:04.342976 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:04.342989 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:04.343000 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:04.343273 (+ 273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:04.343765 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:04.343773 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:04.343779 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:04.344531 (+ 752us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:04.344732 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:04.345547 (+ 815us) server_negotiation.cc:299] Negotiation successful
0810 18:46:04.346022 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3221,"thread_start_us":3119,"threads_started":1}
I20260810 18:46:04.351325 29995 heartbeater.cc:347] Connected to a master server at 127.4.228.254:35121
I20260810 18:46:04.351749 29995 heartbeater.cc:464] Registering TS with master...
I20260810 18:46:04.352777 29995 heartbeater.cc:511] Master 127.4.228.254:35121 requested a full tablet report, sending...
I20260810 18:46:04.356930 29708 ts_manager.cc:194] Registered new tserver with Master: 3864ce298e8744258f3cdc6c68f97864 (127.4.228.193:39905)
I20260810 18:46:04.364068 29708 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.193@KRBTEST.COM'} at 127.4.228.193:47027
Aug 10 18:46:04 dist-test-slave-7kzw krb5kdc[28843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387563, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for kudu/127.4.228.193@KRBTEST.COM
I20260810 18:46:04.378255 30001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:04.361865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:39905 (local address 127.0.0.1:47220)
0810 18:46:04.362534 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:04.362579 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:04.362754 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:04.363941 (+ 1187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:04.363949 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:04.364831 (+ 882us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:46:04.365215 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:04.365229 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:04.366774 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:04.366780 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:04.367984 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:04.368002 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:04.368880 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:04.368888 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:04.369120 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:04.369829 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:46:04.369847 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:46:04.372125 (+ 2278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:46:04.375480 (+ 3355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:46:04.375492 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:46:04.375499 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:46:04.375893 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:46:04.376302 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:46:04.376308 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:46:04.376311 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:46:04.376456 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:46:04.377092 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:46:04.377100 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:46:04.377279 (+ 179us) client_negotiation.cc:769] Sending connection context
0810 18:46:04.377480 (+ 201us) client_negotiation.cc:240] Negotiation successful
0810 18:46:04.377807 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":304,"threads_started":1}
I20260810 18:46:04.378304 30002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:04.362069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47220 (local address 127.4.228.193:39905)
0810 18:46:04.363351 (+ 1282us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:04.363359 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:04.363389 (+ 30us) server_negotiation.cc:407] Connection header received
0810 18:46:04.363471 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:04.363478 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:04.363718 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:04.363878 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:04.365413 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:04.366615 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:04.368140 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:04.368666 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:04.368814 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:04.372352 (+ 3538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:04.372363 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:04.372375 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:46:04.372410 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:04.375333 (+ 2923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:04.376036 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:04.376050 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:04.376054 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:04.376175 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:04.376578 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:04.376583 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:04.376587 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:04.376929 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:04.377135 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:04.377623 (+ 488us) server_negotiation.cc:299] Negotiation successful
0810 18:46:04.377909 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1117,"thread_start_us":961,"threads_started":1}
I20260810 18:46:04.380012 29896 tablet_service.cc:1514] Processing DeleteTablet for tablet 731eb4951a2a4d75ae3dddf6e99c2850 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:45:42 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:47220
I20260810 18:46:04.382125 30004 tablet_replica.cc:330] stopping tablet replica
I20260810 18:46:04.382587 30004 raft_consensus.cc:2229] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:46:04.382997 30004 raft_consensus.cc:2258] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:46:04.384289 30004 ts_tablet_manager.cc:1905] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:46:04.392257 30004 ts_tablet_manager.cc:1918] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:46:04.392439 30004 log.cc:1196] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/wal/wals/731eb4951a2a4d75ae3dddf6e99c2850
I20260810 18:46:04.392938 30004 ts_tablet_manager.cc:1939] T 731eb4951a2a4d75ae3dddf6e99c2850 P 3864ce298e8744258f3cdc6c68f97864: Deleting consensus metadata
I20260810 18:46:04.395406 29686 catalog_manager.cc:4841] TS 3864ce298e8744258f3cdc6c68f97864 (127.4.228.193:39905): tablet 731eb4951a2a4d75ae3dddf6e99c2850 (table test-table [id=d64c88434fb04351ab5b5af3923d74e0]) successfully deleted
W20260810 18:46:04.689952 30000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:46:04.690234 30000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:46:04.690398 30000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:46:04.697856 30000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:46:04.697962 30000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:46:04.698074 30000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.194
I20260810 18:46:04.708045 30000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:34661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.194:45331
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.228.194
--webserver_port=37951
--tserver_master_addrs=127.4.228.254:35121
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.194
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:46:04.709396 30000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:46:04.710865 30000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:46:04.724871 30013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:04.725052 30016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:04.724865 30019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:46:04.726068 30000 server_base.cc:1034] running on GCE node
I20260810 18:46:04.726804 30000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:46:04.727830 30000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:46:04.729081 30000 hybrid_clock.cc:648] HybridClock initialized: now 1786387564729044 us; error 65 us; skew 500 ppm
I20260810 18:46:04.729440 30000 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:46:04 dist-test-slave-7kzw krb5kdc[28843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387564, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:46:04.732879 30000 init.cc:377] Logged in from keytab as kudu/127.4.228.194@KRBTEST.COM (short username kudu)
I20260810 18:46:04.734414 30000 webserver.cc:460] Webserver started at http://127.4.228.194:37951/ using document root <none> and password file <none>
I20260810 18:46:04.735183 30000 fs_manager.cc:359] Metadata directory not provided
I20260810 18:46:04.735359 30000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:46:04.741011 30000 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.005s
I20260810 18:46:04.744022 30029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:04.745389 30000 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:46:04.745582 30000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/wal
uuid: "c04280931825462796891c073e8074ac"
format_stamp: "Formatted at 2026-08-10 18:45:41 on dist-test-slave-7kzw"
server_key: "3633085410b20a64de2bd071215e3de3"
server_key_iv: "1e79c438679ed9c0df5d9d902c593543"
server_key_version: "encryptionkey@0"
I20260810 18:46:04.746563 30000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:46:04.765226 30000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:46:04.769425 30000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:46:04.769726 30000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:46:04.770800 30000 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:46:04.772202 30000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:46:04.772292 30000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:04.772467 30000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:46:04.772562 30000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:04.807070 30000 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.194:45331
I20260810 18:46:04.807133 30210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.194:45331 every 8 connection(s)
I20260810 18:46:04.808667 30000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
I20260810 18:46:04.811574 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 30000
I20260810 18:46:04.812300 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.195:40527
--local_ip_for_outbound_sockets=127.4.228.195
--tserver_master_addrs=127.4.228.254:35121
--webserver_port=42191
--webserver_interface=127.4.228.195
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:34661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:46:04 dist-test-slave-7kzw krb5kdc[28843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387564, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:46:04.833137 29997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:04.811647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:37723 (local address 127.4.228.254:35121)
0810 18:46:04.811903 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:04.811911 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:04.813440 (+ 1529us) server_negotiation.cc:407] Connection header received
0810 18:46:04.815463 (+ 2023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:04.815472 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:04.815592 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:04.815729 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:04.817354 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:04.818600 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:04.820546 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:04.821284 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:04.821419 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:04.825829 (+ 4410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:04.825834 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:04.825838 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:46:04.825875 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:04.829552 (+ 3677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:04.830324 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:04.830333 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:04.830339 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:04.830570 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:04.831023 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:04.831030 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:04.831036 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:04.831454 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:04.831603 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:04.832345 (+ 742us) server_negotiation.cc:299] Negotiation successful
0810 18:46:04.832646 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260810 18:46:04.833812 30213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:04.812112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.194:37723)
0810 18:46:04.813233 (+ 1121us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:04.813297 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:04.815161 (+ 1864us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:04.816009 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:04.816018 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:04.816550 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:46:04.817143 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:04.817161 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:04.818774 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:04.818778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:04.820281 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:04.820300 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:04.821583 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:04.821593 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:04.821824 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:04.822602 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:46:04.822626 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:46:04.825573 (+ 2947us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:46:04.829706 (+ 4133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:46:04.829713 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:46:04.829727 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:46:04.830179 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:46:04.830694 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:46:04.830697 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:46:04.830699 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:46:04.830902 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:46:04.831594 (+ 692us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:46:04.831600 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:46:04.831938 (+ 338us) client_negotiation.cc:769] Sending connection context
0810 18:46:04.832192 (+ 254us) client_negotiation.cc:240] Negotiation successful
0810 18:46:04.832586 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":814,"thread_start_us":619,"threads_started":1}
I20260810 18:46:04.835579 30211 heartbeater.cc:347] Connected to a master server at 127.4.228.254:35121
I20260810 18:46:04.835937 30211 heartbeater.cc:464] Registering TS with master...
I20260810 18:46:04.836947 30211 heartbeater.cc:511] Master 127.4.228.254:35121 requested a full tablet report, sending...
I20260810 18:46:04.838795 29708 ts_manager.cc:194] Registered new tserver with Master: c04280931825462796891c073e8074ac (127.4.228.194:45331)
I20260810 18:46:04.839689 29708 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.194@KRBTEST.COM'} at 127.4.228.194:37723
W20260810 18:46:05.172204 30216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:46:05.172539 30216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:46:05.172720 30216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:46:05.180171 30216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:46:05.180363 30216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:46:05.180557 30216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.195
I20260810 18:46:05.190922 30216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:34661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.195:40527
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.228.195
--webserver_port=42191
--tserver_master_addrs=127.4.228.254:35121
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.195
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:46:05.192332 30216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:46:05.193753 30216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:46:05.207757 30229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:05.207718 30232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:05.207603 30228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:46:05.209313 30216 server_base.cc:1034] running on GCE node
I20260810 18:46:05.209934 30216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:46:05.211105 30216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:46:05.212359 30216 hybrid_clock.cc:648] HybridClock initialized: now 1786387565212312 us; error 43 us; skew 500 ppm
I20260810 18:46:05.212757 30216 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:46:05 dist-test-slave-7kzw krb5kdc[28843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387565, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:46:05.216434 30216 init.cc:377] Logged in from keytab as kudu/127.4.228.195@KRBTEST.COM (short username kudu)
I20260810 18:46:05.218062 30216 webserver.cc:460] Webserver started at http://127.4.228.195:42191/ using document root <none> and password file <none>
I20260810 18:46:05.218876 30216 fs_manager.cc:359] Metadata directory not provided
I20260810 18:46:05.219044 30216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:46:05.224812 30216 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:46:05.227711 30241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:05.229288 30216 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:46:05.229491 30216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/wal
uuid: "23521ea509214ca89745701374ff8eda"
format_stamp: "Formatted at 2026-08-10 18:45:42 on dist-test-slave-7kzw"
server_key: "8d48b38acf11af152cf35838bc180760"
server_key_iv: "912a5d8864142603f986f7f3dcf7d872"
server_key_version: "encryptionkey@0"
I20260810 18:46:05.230525 30216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:46:05.263140 30216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:46:05.267287 30216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:46:05.267580 30216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:46:05.268627 30216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:46:05.270102 30216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:46:05.270234 30216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:05.270372 30216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:46:05.270468 30216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:05.305930 30216 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.195:40527
I20260810 18:46:05.306035 30417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.195:40527 every 8 connection(s)
I20260810 18:46:05.307543 30216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
I20260810 18:46:05.310083 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 30216
Aug 10 18:46:05 dist-test-slave-7kzw krb5kdc[28843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387565, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:46:05.329504 30422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:05.311224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44242 (local address 127.4.228.254:35121)
0810 18:46:05.311819 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:05.311828 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:05.312398 (+ 570us) server_negotiation.cc:407] Connection header received
0810 18:46:05.312656 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:05.312662 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:05.312748 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:05.312872 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:05.315230 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:05.316157 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:05.318154 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:05.318805 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:05.318948 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:05.322789 (+ 3841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:05.322798 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:05.322803 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:46:05.322843 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:05.326028 (+ 3185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:05.326995 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:05.327001 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:05.327004 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:05.327235 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:05.327732 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:05.327741 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:05.327746 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:05.328147 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:05.328450 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:05.328829 (+ 379us) server_negotiation.cc:299] Negotiation successful
0810 18:46:05.329104 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":400,"thread_start_us":266,"threads_started":1}
I20260810 18:46:05.333912 29997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:05.310571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:47491 (local address 127.4.228.254:35121)
0810 18:46:05.310760 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:05.310765 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:05.312997 (+ 2232us) server_negotiation.cc:407] Connection header received
0810 18:46:05.315734 (+ 2737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:05.315739 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:05.315809 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:05.315908 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:05.317899 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:05.319107 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:05.321033 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:05.321467 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:05.321603 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:05.326109 (+ 4506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:05.326117 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:05.326122 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:46:05.326177 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:05.330069 (+ 3892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:05.331460 (+ 1391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:05.331465 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:05.331468 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:05.331609 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:05.332092 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:05.332102 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:05.332104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:05.332461 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:05.332630 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:05.333308 (+ 678us) server_negotiation.cc:299] Negotiation successful
0810 18:46:05.333562 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":61}
I20260810 18:46:05.335062 30427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:05.311200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.195:47491)
0810 18:46:05.312811 (+ 1611us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:05.312890 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:05.315441 (+ 2551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:05.316113 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:05.316121 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:05.316926 (+ 805us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:46:05.317695 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:05.317723 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:05.319263 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:05.319269 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:05.320879 (+ 1610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:05.320892 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:05.321715 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:05.321725 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:05.321957 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:05.322654 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:46:05.322688 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:46:05.325862 (+ 3174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:46:05.330221 (+ 4359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:46:05.330230 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:46:05.330247 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:46:05.331331 (+ 1084us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:46:05.331759 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:46:05.331763 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:46:05.331765 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:46:05.331949 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:46:05.332599 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:46:05.332608 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:46:05.332940 (+ 332us) client_negotiation.cc:769] Sending connection context
0810 18:46:05.333212 (+ 272us) client_negotiation.cc:240] Negotiation successful
0810 18:46:05.333757 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1283,"thread_start_us":1048,"threads_started":1}
I20260810 18:46:05.336536 30418 heartbeater.cc:347] Connected to a master server at 127.4.228.254:35121
I20260810 18:46:05.336858 30418 heartbeater.cc:464] Registering TS with master...
I20260810 18:46:05.337581 30418 heartbeater.cc:511] Master 127.4.228.254:35121 requested a full tablet report, sending...
I20260810 18:46:05.339241 29708 ts_manager.cc:194] Registered new tserver with Master: 23521ea509214ca89745701374ff8eda (127.4.228.195:40527)
I20260810 18:46:05.340147 29708 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.195@KRBTEST.COM'} at 127.4.228.195:47491
I20260810 18:46:05.344843 5011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:46:05.352999 29997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:05.345918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44246 (local address 127.4.228.254:35121)
0810 18:46:05.346094 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:05.346099 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:05.346744 (+ 645us) server_negotiation.cc:407] Connection header received
0810 18:46:05.346972 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:05.346977 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:05.347154 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:05.347316 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:46:05.347761 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:05.348571 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:05.350319 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:05.350874 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:05.351021 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:05.351819 (+ 798us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:46:05.352163 (+ 344us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0810 18:46:05.352709 (+ 546us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44246: token expired
Metrics: {"server-negotiator.queue_time_us":61}
W20260810 18:46:05.353161 29997 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44246: token expired
W20260810 18:46:05.353940 29594 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.4.228.254:35121)
I20260810 18:46:05.367712 29997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:05.354790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44256 (local address 127.4.228.254:35121)
0810 18:46:05.354985 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:05.354992 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:05.355008 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:46:05.355051 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:05.355055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:05.355139 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:05.355247 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:05.356696 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:05.357389 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:05.358876 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:05.359295 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:05.359435 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:05.362247 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:05.362252 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:05.362255 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:46:05.362288 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:05.364873 (+ 2585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:05.365615 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:05.365632 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:05.365635 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:05.365775 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:05.366167 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:05.366174 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:05.366177 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:05.366450 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:05.366774 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:05.367125 (+ 351us) server_negotiation.cc:299] Negotiation successful
0810 18:46:05.367366 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 18:46:05.368148 29995 heartbeater.cc:503] Master 127.4.228.254:35121 was elected leader, sending a full tablet report...
I20260810 18:46:05.371598 29708 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44256:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:46:05.379571 29708 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:46:05.392931 29896 tablet_service.cc:1467] Processing CreateTablet for tablet 3c0546f9d34c4f56a2c66508ec6e458d (DEFAULT_TABLE table=test-table [id=8a28b9afee0d4226b847e1323a9fbda7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:46:05.393824 29896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c0546f9d34c4f56a2c66508ec6e458d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:46:05.405156 30441 tablet_bootstrap.cc:492] T 3c0546f9d34c4f56a2c66508ec6e458d P 3864ce298e8744258f3cdc6c68f97864: Bootstrap starting.
I20260810 18:46:05.407466 30441 tablet_bootstrap.cc:654] T 3c0546f9d34c4f56a2c66508ec6e458d P 3864ce298e8744258f3cdc6c68f97864: Neither blocks nor log segments found. Creating new log.
I20260810 18:46:05.409956 30441 tablet_bootstrap.cc:492] T 3c0546f9d34c4f56a2c66508ec6e458d P 3864ce298e8744258f3cdc6c68f97864: No bootstrap required, opened a new log
I20260810 18:46:05.410127 30441 ts_tablet_manager.cc:1397] T 3c0546f9d34c4f56a2c66508ec6e458d P 3864ce298e8744258f3cdc6c68f97864: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 18:46:05.411051 30441 raft_consensus.cc:348] T 3c0546f9d34c4f56a2c66508ec6e458d P 3864ce298e8744258f3cdc6c68f97864 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c04280931825462796891c073e8074ac" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 45331 } } peers { permanent_uuid: "23521ea509214ca89745701374ff8eda" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40527 } } peers { permanent_uuid: "3864ce298e8744258f3cdc6c68f97864" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 39905 } }
I20260810 18:46:05.411339 30441 raft_consensus.cc:374] T 3c0546f9d34c4f56a2c66508ec6e458d P 3864ce298e8744258f3cdc6c68f97864 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:46:05.411454 30441 raft_consensus.cc:729] T 3c0546f9d34c4f56a2c66508ec6e458d P 3864ce298e8744258f3cdc6c68f97864 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3864ce298e8744258f3cdc6c68f97864, State: Initialized, Role: FOLLOWER
I20260810 18:46:05.411782 30441 consensus_queue.cc:260] T 3c0546f9d34c4f56a2c66508ec6e458d P 3864ce298e8744258f3cdc6c68f97864 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c04280931825462796891c073e8074ac" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 45331 } } peers { permanent_uuid: "23521ea509214ca89745701374ff8eda" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40527 } } peers { permanent_uuid: "3864ce298e8744258f3cdc6c68f97864" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 39905 } }
I20260810 18:46:05.411996 30433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:05.390765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:45331 (local address 127.0.0.1:53924)
0810 18:46:05.392461 (+ 1696us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:05.392485 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:05.392779 (+ 294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:05.403965 (+ 11186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:05.403971 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:05.404002 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:46:05.404463 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:05.404482 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:05.406640 (+ 2158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:05.406648 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:05.408956 (+ 2308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:05.408969 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:05.411024 (+ 2055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:05.411033 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:05.411229 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:05.411292 (+ 63us) client_negotiation.cc:769] Sending connection context
0810 18:46:05.411408 (+ 116us) client_negotiation.cc:240] Negotiation successful
0810 18:46:05.411570 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1539,"thread_start_us":3836,"threads_started":1}
I20260810 18:46:05.413314 30434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:05.390990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53924 (local address 127.4.228.194:45331)
0810 18:46:05.401998 (+ 11008us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:05.402010 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:05.402046 (+ 36us) server_negotiation.cc:407] Connection header received
0810 18:46:05.402175 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:05.402185 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:05.402519 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:05.403968 (+ 1449us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:46:05.404661 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:05.406203 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:05.409129 (+ 2926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:05.410733 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:05.410883 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:05.411106 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:05.412586 (+ 1480us) server_negotiation.cc:299] Negotiation successful
0810 18:46:05.412864 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10795,"thread_start_us":11598,"threads_started":1}
I20260810 18:46:05.415158 30440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:05.391803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59390 (local address 127.4.228.195:40527)
0810 18:46:05.405753 (+ 13950us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:05.405762 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:05.406420 (+ 658us) server_negotiation.cc:407] Connection header received
0810 18:46:05.406838 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:05.406848 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:05.407108 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:05.407251 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:46:05.408233 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:05.409749 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:05.412385 (+ 2636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:05.413816 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:05.413995 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:05.414251 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:05.414597 (+ 346us) server_negotiation.cc:299] Negotiation successful
0810 18:46:05.414780 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13753,"thread_start_us":13517,"threads_started":1}
I20260810 18:46:05.415689 30436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:05.391412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:40527 (local address 127.0.0.1:59390)
0810 18:46:05.406280 (+ 14868us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:05.406310 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:05.406587 (+ 277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:05.407610 (+ 1023us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:05.407619 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:05.407647 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:46:05.408027 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:05.408044 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:05.409918 (+ 1874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:05.409926 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:05.412167 (+ 2241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:05.412186 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:05.414115 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:05.414124 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:05.414323 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:05.414351 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 18:46:05.414481 (+ 130us) client_negotiation.cc:240] Negotiation successful
0810 18:46:05.414657 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14683,"thread_start_us":14895,"threads_started":1}
I20260810 18:46:05.417884 30109 tablet_service.cc:1467] Processing CreateTablet for tablet 3c0546f9d34c4f56a2c66508ec6e458d (DEFAULT_TABLE table=test-table [id=8a28b9afee0d4226b847e1323a9fbda7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:46:05.419435 30109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c0546f9d34c4f56a2c66508ec6e458d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:46:05.419494 30316 tablet_service.cc:1467] Processing CreateTablet for tablet 3c0546f9d34c4f56a2c66508ec6e458d (DEFAULT_TABLE table=test-table [id=8a28b9afee0d4226b847e1323a9fbda7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:46:05.420837 30316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c0546f9d34c4f56a2c66508ec6e458d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:46:05.424106 30441 ts_tablet_manager.cc:1428] T 3c0546f9d34c4f56a2c66508ec6e458d P 3864ce298e8744258f3cdc6c68f97864: Time spent starting tablet: real 0.014s user 0.002s sys 0.003s
I20260810 18:46:05.432539 30446 tablet_bootstrap.cc:492] T 3c0546f9d34c4f56a2c66508ec6e458d P c04280931825462796891c073e8074ac: Bootstrap starting.
I20260810 18:46:05.434540 30448 tablet_bootstrap.cc:492] T 3c0546f9d34c4f56a2c66508ec6e458d P 23521ea509214ca89745701374ff8eda: Bootstrap starting.
I20260810 18:46:05.436245 30446 tablet_bootstrap.cc:654] T 3c0546f9d34c4f56a2c66508ec6e458d P c04280931825462796891c073e8074ac: Neither blocks nor log segments found. Creating new log.
I20260810 18:46:05.437318 30446 log.cc:824] T 3c0546f9d34c4f56a2c66508ec6e458d P c04280931825462796891c073e8074ac: Log is configured to *not* fsync() on all Append() calls
I20260810 18:46:05.437841 30448 tablet_bootstrap.cc:654] T 3c0546f9d34c4f56a2c66508ec6e458d P 23521ea509214ca89745701374ff8eda: Neither blocks nor log segments found. Creating new log.
I20260810 18:46:05.439042 30448 log.cc:824] T 3c0546f9d34c4f56a2c66508ec6e458d P 23521ea509214ca89745701374ff8eda: Log is configured to *not* fsync() on all Append() calls
I20260810 18:46:05.440445 30446 tablet_bootstrap.cc:492] T 3c0546f9d34c4f56a2c66508ec6e458d P c04280931825462796891c073e8074ac: No bootstrap required, opened a new log
I20260810 18:46:05.440724 30446 ts_tablet_manager.cc:1397] T 3c0546f9d34c4f56a2c66508ec6e458d P c04280931825462796891c073e8074ac: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260810 18:46:05.441916 30448 tablet_bootstrap.cc:492] T 3c0546f9d34c4f56a2c66508ec6e458d P 23521ea509214ca89745701374ff8eda: No bootstrap required, opened a new log
I20260810 18:46:05.442224 30448 ts_tablet_manager.cc:1397] T 3c0546f9d34c4f56a2c66508ec6e458d P 23521ea509214ca89745701374ff8eda: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260810 18:46:05.448376 30446 raft_consensus.cc:348] T 3c0546f9d34c4f56a2c66508ec6e458d P c04280931825462796891c073e8074ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c04280931825462796891c073e8074ac" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 45331 } } peers { permanent_uuid: "23521ea509214ca89745701374ff8eda" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40527 } } peers { permanent_uuid: "3864ce298e8744258f3cdc6c68f97864" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 39905 } }
I20260810 18:46:05.448745 30446 raft_consensus.cc:374] T 3c0546f9d34c4f56a2c66508ec6e458d P c04280931825462796891c073e8074ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:46:05.448872 30446 raft_consensus.cc:729] T 3c0546f9d34c4f56a2c66508ec6e458d P c04280931825462796891c073e8074ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c04280931825462796891c073e8074ac, State: Initialized, Role: FOLLOWER
I20260810 18:46:05.449548 30446 consensus_queue.cc:260] T 3c0546f9d34c4f56a2c66508ec6e458d P c04280931825462796891c073e8074ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c04280931825462796891c073e8074ac" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 45331 } } peers { permanent_uuid: "23521ea509214ca89745701374ff8eda" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40527 } } peers { permanent_uuid: "3864ce298e8744258f3cdc6c68f97864" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 39905 } }
I20260810 18:46:05.450861 30211 heartbeater.cc:503] Master 127.4.228.254:35121 was elected leader, sending a full tablet report...
I20260810 18:46:05.450992 30448 raft_consensus.cc:348] T 3c0546f9d34c4f56a2c66508ec6e458d P 23521ea509214ca89745701374ff8eda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c04280931825462796891c073e8074ac" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 45331 } } peers { permanent_uuid: "23521ea509214ca89745701374ff8eda" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40527 } } peers { permanent_uuid: "3864ce298e8744258f3cdc6c68f97864" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 39905 } }
I20260810 18:46:05.451491 30448 raft_consensus.cc:374] T 3c0546f9d34c4f56a2c66508ec6e458d P 23521ea509214ca89745701374ff8eda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:46:05.451529 30446 ts_tablet_manager.cc:1428] T 3c0546f9d34c4f56a2c66508ec6e458d P c04280931825462796891c073e8074ac: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260810 18:46:05.451610 30448 raft_consensus.cc:729] T 3c0546f9d34c4f56a2c66508ec6e458d P 23521ea509214ca89745701374ff8eda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23521ea509214ca89745701374ff8eda, State: Initialized, Role: FOLLOWER
I20260810 18:46:05.452289 30448 consensus_queue.cc:260] T 3c0546f9d34c4f56a2c66508ec6e458d P 23521ea509214ca89745701374ff8eda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c04280931825462796891c073e8074ac" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 45331 } } peers { permanent_uuid: "23521ea509214ca89745701374ff8eda" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40527 } } peers { permanent_uuid: "3864ce298e8744258f3cdc6c68f97864" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 39905 } }
I20260810 18:46:05.453429 30418 heartbeater.cc:503] Master 127.4.228.254:35121 was elected leader, sending a full tablet report...
I20260810 18:46:05.454279 30448 ts_tablet_manager.cc:1428] T 3c0546f9d34c4f56a2c66508ec6e458d P 23521ea509214ca89745701374ff8eda: Time spent starting tablet: real 0.012s user 0.009s sys 0.004s
I20260810 18:46:05.489341 30444 raft_consensus.cc:482] T 3c0546f9d34c4f56a2c66508ec6e458d P 3864ce298e8744258f3cdc6c68f97864 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:46:05.489821 30444 raft_consensus.cc:504] T 3c0546f9d34c4f56a2c66508ec6e458d P 3864ce298e8744258f3cdc6c68f97864 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c04280931825462796891c073e8074ac" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 45331 } } peers { permanent_uuid: "23521ea509214ca89745701374ff8eda" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40527 } } peers { permanent_uuid: "3864ce298e8744258f3cdc6c68f97864" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 39905 } }
I20260810 18:46:05.491976 30444 leader_election.cc:288] T 3c0546f9d34c4f56a2c66508ec6e458d P 3864ce298e8744258f3cdc6c68f97864 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c04280931825462796891c073e8074ac (127.4.228.194:45331), 23521ea509214ca89745701374ff8eda (127.4.228.195:40527)
I20260810 18:46:05.499862 30440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:05.492851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:51933 (local address 127.4.228.195:40527)
0810 18:46:05.493080 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:05.493086 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:05.493213 (+ 127us) server_negotiation.cc:407] Connection header received
0810 18:46:05.493523 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:05.493529 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:05.493632 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:05.493752 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:46:05.494452 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:05.495602 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:05.497493 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:05.498771 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:05.498905 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:05.498964 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:05.499364 (+ 400us) server_negotiation.cc:299] Negotiation successful
0810 18:46:05.499545 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 18:46:05.499863 30459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:05.492278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:40527 (local address 127.4.228.193:51933)
0810 18:46:05.493047 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:05.493074 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:05.493307 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:05.493792 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:05.493796 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:05.493825 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:46:05.494295 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:05.494311 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:05.495748 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:05.495754 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:05.497314 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:05.497327 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:05.498959 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:05.498966 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:05.499142 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:05.499207 (+ 65us) client_negotiation.cc:769] Sending connection context
0810 18:46:05.499324 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 18:46:05.499468 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"spinlock_wait_cycles":26496,"thread_start_us":310,"threads_started":1}
I20260810 18:46:05.500373 30434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:05.492380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:41395 (local address 127.4.228.194:45331)
0810 18:46:05.492658 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:05.492666 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:05.493804 (+ 1138us) server_negotiation.cc:407] Connection header received
0810 18:46:05.494157 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:05.494163 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:05.494248 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:05.494320 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:46:05.495251 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:05.496118 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:05.498295 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:05.499339 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:05.499476 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:05.499541 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:05.500005 (+ 464us) server_negotiation.cc:299] Negotiation successful
0810 18:46:05.500139 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 18:46:05.500699 30461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:05.492284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:45331 (local address 127.4.228.193:41395)
0810 18:46:05.493692 (+ 1408us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:05.493721 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:05.493930 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:05.494569 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:05.494575 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:05.494608 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:46:05.495047 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:05.495059 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:05.496274 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:05.496295 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:05.498113 (+ 1818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:05.498130 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:05.499560 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:05.499568 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:05.499782 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:05.499821 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 18:46:05.500125 (+ 304us) client_negotiation.cc:240] Negotiation successful
0810 18:46:05.500287 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1240,"mutex_wait_us":5,"thread_start_us":957,"threads_started":1}
I20260810 18:46:05.501134 30345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c0546f9d34c4f56a2c66508ec6e458d" candidate_uuid: "3864ce298e8744258f3cdc6c68f97864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23521ea509214ca89745701374ff8eda" is_pre_election: true
I20260810 18:46:05.501837 30345 raft_consensus.cc:2454] T 3c0546f9d34c4f56a2c66508ec6e458d P 23521ea509214ca89745701374ff8eda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3864ce298e8744258f3cdc6c68f97864 in term 0.
I20260810 18:46:05.501849 30141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c0546f9d34c4f56a2c66508ec6e458d" candidate_uuid: "3864ce298e8744258f3cdc6c68f97864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c04280931825462796891c073e8074ac" is_pre_election: true
I20260810 18:46:05.502434 30141 raft_consensus.cc:2454] T 3c0546f9d34c4f56a2c66508ec6e458d P c04280931825462796891c073e8074ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3864ce298e8744258f3cdc6c68f97864 in term 0.
I20260810 18:46:05.502741 29830 leader_election.cc:302] T 3c0546f9d34c4f56a2c66508ec6e458d P 3864ce298e8744258f3cdc6c68f97864 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23521ea509214ca89745701374ff8eda, 3864ce298e8744258f3cdc6c68f97864; no voters:
I20260810 18:46:05.503263 30444 raft_consensus.cc:2781] T 3c0546f9d34c4f56a2c66508ec6e458d P 3864ce298e8744258f3cdc6c68f97864 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:46:05.503439 30444 raft_consensus.cc:482] T 3c0546f9d34c4f56a2c66508ec6e458d P 3864ce298e8744258f3cdc6c68f97864 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:46:05.503568 30444 raft_consensus.cc:3037] T 3c0546f9d34c4f56a2c66508ec6e458d P 3864ce298e8744258f3cdc6c68f97864 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:46:05.506016 30444 raft_consensus.cc:504] T 3c0546f9d34c4f56a2c66508ec6e458d P 3864ce298e8744258f3cdc6c68f97864 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c04280931825462796891c073e8074ac" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 45331 } } peers { permanent_uuid: "23521ea509214ca89745701374ff8eda" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40527 } } peers { permanent_uuid: "3864ce298e8744258f3cdc6c68f97864" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 39905 } }
I20260810 18:46:05.506789 30444 leader_election.cc:288] T 3c0546f9d34c4f56a2c66508ec6e458d P 3864ce298e8744258f3cdc6c68f97864 [CANDIDATE]: Term 1 election: Requested vote from peers c04280931825462796891c073e8074ac (127.4.228.194:45331), 23521ea509214ca89745701374ff8eda (127.4.228.195:40527)
I20260810 18:46:05.507323 30141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c0546f9d34c4f56a2c66508ec6e458d" candidate_uuid: "3864ce298e8744258f3cdc6c68f97864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c04280931825462796891c073e8074ac"
I20260810 18:46:05.507411 30345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c0546f9d34c4f56a2c66508ec6e458d" candidate_uuid: "3864ce298e8744258f3cdc6c68f97864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23521ea509214ca89745701374ff8eda"
I20260810 18:46:05.507555 30141 raft_consensus.cc:3037] T 3c0546f9d34c4f56a2c66508ec6e458d P c04280931825462796891c073e8074ac [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:46:05.507704 30345 raft_consensus.cc:3037] T 3c0546f9d34c4f56a2c66508ec6e458d P 23521ea509214ca89745701374ff8eda [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:46:05.510046 30345 raft_consensus.cc:2454] T 3c0546f9d34c4f56a2c66508ec6e458d P 23521ea509214ca89745701374ff8eda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3864ce298e8744258f3cdc6c68f97864 in term 1.
I20260810 18:46:05.510144 30141 raft_consensus.cc:2454] T 3c0546f9d34c4f56a2c66508ec6e458d P c04280931825462796891c073e8074ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3864ce298e8744258f3cdc6c68f97864 in term 1.
I20260810 18:46:05.510603 29830 leader_election.cc:302] T 3c0546f9d34c4f56a2c66508ec6e458d P 3864ce298e8744258f3cdc6c68f97864 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23521ea509214ca89745701374ff8eda, 3864ce298e8744258f3cdc6c68f97864; no voters:
I20260810 18:46:05.510965 30444 raft_consensus.cc:2781] T 3c0546f9d34c4f56a2c66508ec6e458d P 3864ce298e8744258f3cdc6c68f97864 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:46:05.511505 30444 raft_consensus.cc:686] T 3c0546f9d34c4f56a2c66508ec6e458d P 3864ce298e8744258f3cdc6c68f97864 [term 1 LEADER]: Becoming Leader. State: Replica: 3864ce298e8744258f3cdc6c68f97864, State: Running, Role: LEADER
I20260810 18:46:05.512004 30444 consensus_queue.cc:237] T 3c0546f9d34c4f56a2c66508ec6e458d P 3864ce298e8744258f3cdc6c68f97864 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c04280931825462796891c073e8074ac" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 45331 } } peers { permanent_uuid: "23521ea509214ca89745701374ff8eda" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40527 } } peers { permanent_uuid: "3864ce298e8744258f3cdc6c68f97864" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 39905 } }
I20260810 18:46:05.517242 29708 catalog_manager.cc:5495] T 3c0546f9d34c4f56a2c66508ec6e458d P 3864ce298e8744258f3cdc6c68f97864 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3864ce298e8744258f3cdc6c68f97864 (127.4.228.193). New cstate: current_term: 1 leader_uuid: "3864ce298e8744258f3cdc6c68f97864" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c04280931825462796891c073e8074ac" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 45331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23521ea509214ca89745701374ff8eda" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3864ce298e8744258f3cdc6c68f97864" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 39905 } health_report { overall_health: HEALTHY } } }
W20260810 18:46:05.559240 30421 tablet.cc:2369] T 3c0546f9d34c4f56a2c66508ec6e458d P 23521ea509214ca89745701374ff8eda: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:46:05.560794 30212 tablet.cc:2369] T 3c0546f9d34c4f56a2c66508ec6e458d P c04280931825462796891c073e8074ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:46:05.563458 30465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:05.556579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47234 (local address 127.4.228.193:39905)
0810 18:46:05.557298 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:05.557303 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:05.557326 (+ 23us) server_negotiation.cc:407] Connection header received
0810 18:46:05.557423 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:05.557427 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:05.557509 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:05.557709 (+ 200us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:46:05.558260 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:05.559172 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:05.561049 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:05.561515 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:05.561623 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:05.562155 (+ 532us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:46:05.562830 (+ 675us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:46:05.562950 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:05.563097 (+ 147us) server_negotiation.cc:299] Negotiation successful
0810 18:46:05.563168 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":424,"threads_started":1}
I20260810 18:46:05.569669 30141 raft_consensus.cc:1261] T 3c0546f9d34c4f56a2c66508ec6e458d P c04280931825462796891c073e8074ac [term 1 FOLLOWER]: Refusing update from remote peer 3864ce298e8744258f3cdc6c68f97864: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:46:05.569720 30345 raft_consensus.cc:1261] T 3c0546f9d34c4f56a2c66508ec6e458d P 23521ea509214ca89745701374ff8eda [term 1 FOLLOWER]: Refusing update from remote peer 3864ce298e8744258f3cdc6c68f97864: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:46:05.570748 30462 consensus_queue.cc:1035] T 3c0546f9d34c4f56a2c66508ec6e458d P 3864ce298e8744258f3cdc6c68f97864 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c04280931825462796891c073e8074ac" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 45331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:46:05.571070 30444 consensus_queue.cc:1035] T 3c0546f9d34c4f56a2c66508ec6e458d P 3864ce298e8744258f3cdc6c68f97864 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23521ea509214ca89745701374ff8eda" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 40527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 18:46:05.573084 29996 tablet.cc:2369] T 3c0546f9d34c4f56a2c66508ec6e458d P 3864ce298e8744258f3cdc6c68f97864: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:46:05.582085 30466 mvcc.cc:205] Tried to move back new op lower bound from 7317043468563902464 to 7317043468342738944. Current Snapshot: MvccSnapshot[applied={T|T < 7317043468563902464}]
I20260810 18:46:05.582716 30469 mvcc.cc:205] Tried to move back new op lower bound from 7317043468563902464 to 7317043468342738944. Current Snapshot: MvccSnapshot[applied={T|T < 7317043468563902464}]
I20260810 18:46:05.586642 30467 mvcc.cc:205] Tried to move back new op lower bound from 7317043468563902464 to 7317043468342738944. Current Snapshot: MvccSnapshot[applied={T|T < 7317043468563902464}]
I20260810 18:46:05.600268 29708 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:46:05.600562 29708 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:46:05.604807 29708 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca: Sending DeleteTablet for 3 replicas of tablet 3c0546f9d34c4f56a2c66508ec6e458d
I20260810 18:46:05.605816 29896 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c0546f9d34c4f56a2c66508ec6e458d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:46:05 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:47220
I20260810 18:46:05.605904 30109 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c0546f9d34c4f56a2c66508ec6e458d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:46:05 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:53924
I20260810 18:46:05.605974 30316 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c0546f9d34c4f56a2c66508ec6e458d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:46:05 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:59390
I20260810 18:46:05.606930 30481 tablet_replica.cc:330] T 3c0546f9d34c4f56a2c66508ec6e458d P 3864ce298e8744258f3cdc6c68f97864: stopping tablet replica
I20260810 18:46:05.607293 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 29774
I20260810 18:46:05.607299 30481 raft_consensus.cc:2229] T 3c0546f9d34c4f56a2c66508ec6e458d P 3864ce298e8744258f3cdc6c68f97864 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:46:05.608187 30481 raft_consensus.cc:2258] T 3c0546f9d34c4f56a2c66508ec6e458d P 3864ce298e8744258f3cdc6c68f97864 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:46:05.608569 30484 tablet_replica.cc:330] T 3c0546f9d34c4f56a2c66508ec6e458d P 23521ea509214ca89745701374ff8eda: stopping tablet replica
I20260810 18:46:05.609185 30484 raft_consensus.cc:2229] T 3c0546f9d34c4f56a2c66508ec6e458d P 23521ea509214ca89745701374ff8eda [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:46:05.609849 30484 raft_consensus.cc:2258] T 3c0546f9d34c4f56a2c66508ec6e458d P 23521ea509214ca89745701374ff8eda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:46:05.610353 30481 ts_tablet_manager.cc:1905] T 3c0546f9d34c4f56a2c66508ec6e458d P 3864ce298e8744258f3cdc6c68f97864: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:46:05.610761 30486 tablet_replica.cc:330] T 3c0546f9d34c4f56a2c66508ec6e458d P c04280931825462796891c073e8074ac: stopping tablet replica
I20260810 18:46:05.611188 30486 raft_consensus.cc:2229] T 3c0546f9d34c4f56a2c66508ec6e458d P c04280931825462796891c073e8074ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:46:05.611945 30486 raft_consensus.cc:2258] T 3c0546f9d34c4f56a2c66508ec6e458d P c04280931825462796891c073e8074ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:46:05.614408 30486 ts_tablet_manager.cc:1905] T 3c0546f9d34c4f56a2c66508ec6e458d P c04280931825462796891c073e8074ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:46:05.614830 30484 ts_tablet_manager.cc:1905] T 3c0546f9d34c4f56a2c66508ec6e458d P 23521ea509214ca89745701374ff8eda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:46:05.621721 30486 ts_tablet_manager.cc:1918] T 3c0546f9d34c4f56a2c66508ec6e458d P c04280931825462796891c073e8074ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:46:05.621922 30486 log.cc:1196] T 3c0546f9d34c4f56a2c66508ec6e458d P c04280931825462796891c073e8074ac: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/wal/wals/3c0546f9d34c4f56a2c66508ec6e458d
I20260810 18:46:05.622539 30486 ts_tablet_manager.cc:1939] T 3c0546f9d34c4f56a2c66508ec6e458d P c04280931825462796891c073e8074ac: Deleting consensus metadata
I20260810 18:46:05.622985 30484 ts_tablet_manager.cc:1918] T 3c0546f9d34c4f56a2c66508ec6e458d P 23521ea509214ca89745701374ff8eda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:46:05.623174 30484 log.cc:1196] T 3c0546f9d34c4f56a2c66508ec6e458d P 23521ea509214ca89745701374ff8eda: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/wal/wals/3c0546f9d34c4f56a2c66508ec6e458d
I20260810 18:46:05.623813 30484 ts_tablet_manager.cc:1939] T 3c0546f9d34c4f56a2c66508ec6e458d P 23521ea509214ca89745701374ff8eda: Deleting consensus metadata
I20260810 18:46:05.624753 29683 catalog_manager.cc:4841] TS c04280931825462796891c073e8074ac (127.4.228.194:45331): tablet 3c0546f9d34c4f56a2c66508ec6e458d (table test-table [id=8a28b9afee0d4226b847e1323a9fbda7]) successfully deleted
I20260810 18:46:05.625869 29684 catalog_manager.cc:4841] TS 23521ea509214ca89745701374ff8eda (127.4.228.195:40527): tablet 3c0546f9d34c4f56a2c66508ec6e458d (table test-table [id=8a28b9afee0d4226b847e1323a9fbda7]) successfully deleted
I20260810 18:46:05.627384 30465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:05.608586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47242 (local address 127.4.228.193:39905)
0810 18:46:05.608935 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:05.608943 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:05.608960 (+ 17us) server_negotiation.cc:407] Connection header received
0810 18:46:05.609019 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:05.609025 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:05.609104 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:05.609215 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:05.611198 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:05.612285 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:05.614150 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:05.614911 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:05.615049 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:05.618801 (+ 3752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:05.618808 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:05.618817 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:46:05.618861 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:05.623483 (+ 4622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:05.624535 (+ 1052us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:05.624541 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:05.624544 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:05.624756 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:05.625288 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:05.625293 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:05.625296 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:05.625723 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:05.626025 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:05.626632 (+ 607us) server_negotiation.cc:299] Negotiation successful
0810 18:46:05.626941 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225}
I20260810 18:46:05.628355 29986 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:46:05.628492 30481 ts_tablet_manager.cc:1918] T 3c0546f9d34c4f56a2c66508ec6e458d P 3864ce298e8744258f3cdc6c68f97864: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:46:05.628643 30481 log.cc:1196] T 3c0546f9d34c4f56a2c66508ec6e458d P 3864ce298e8744258f3cdc6c68f97864: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/wal/wals/3c0546f9d34c4f56a2c66508ec6e458d
I20260810 18:46:05.739893 30481 ts_tablet_manager.cc:1939] T 3c0546f9d34c4f56a2c66508ec6e458d P 3864ce298e8744258f3cdc6c68f97864: Deleting consensus metadata
I20260810 18:46:05.741281 29686 catalog_manager.cc:4841] TS 3864ce298e8744258f3cdc6c68f97864 (127.4.228.193:39905): tablet 3c0546f9d34c4f56a2c66508ec6e458d (table test-table [id=8a28b9afee0d4226b847e1323a9fbda7]) successfully deleted
I20260810 18:46:05.742884 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 29774
I20260810 18:46:05.777475 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 30000
I20260810 18:46:05.792479 30434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:05.778392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53940 (local address 127.4.228.194:45331)
0810 18:46:05.778636 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:05.778643 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:05.778797 (+ 154us) server_negotiation.cc:407] Connection header received
0810 18:46:05.779019 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:05.779024 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:05.779107 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:05.779247 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:05.780659 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:05.781445 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:05.782968 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:05.783395 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:05.783545 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:05.786349 (+ 2804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:05.786363 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:05.786382 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:46:05.786417 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:05.789379 (+ 2962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:05.790224 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:05.790232 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:05.790235 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:05.790369 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:05.790804 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:05.790810 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:05.790813 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:05.791217 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:05.791539 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:05.791900 (+ 361us) server_negotiation.cc:299] Negotiation successful
0810 18:46:05.792145 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260810 18:46:05.793161 30202 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:46:05.885668 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 30000
I20260810 18:46:05.916103 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 30216
I20260810 18:46:05.931772 30440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:05.916962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59392 (local address 127.4.228.195:40527)
0810 18:46:05.917144 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:05.917148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:05.917188 (+ 40us) server_negotiation.cc:407] Connection header received
0810 18:46:05.917272 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:05.917276 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:05.917340 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:05.917446 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:05.919071 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:05.919911 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:05.921355 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:05.921851 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:05.921982 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:05.925337 (+ 3355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:05.925354 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:05.925374 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:46:05.925413 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:05.928474 (+ 3061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:05.929310 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:05.929319 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:05.929322 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:05.929481 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:05.930027 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:05.930031 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:05.930034 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:05.930593 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:05.930902 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:05.931262 (+ 360us) server_negotiation.cc:299] Negotiation successful
0810 18:46:05.931474 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 18:46:05.932592 30411 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:46:06.025533 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 30216
I20260810 18:46:06.054314 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 29658
I20260810 18:46:06.055790 29766 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:46:06.174763 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 29658
I20260810 18:46:26.210582 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:35121
--webserver_interface=127.4.228.254
--webserver_port=34223
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:34661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:35121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:46:26.563089 30493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:46:26.563479 30493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:46:26.563669 30493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:46:26.570904 30493 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 18:46:26.571053 30493 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 18:46:26.571136 30493 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 18:46:26.571216 30493 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:46:26.571331 30493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:46:26.571456 30493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:46:26.571547 30493 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:46:26.571627 30493 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:46:26.581590 30493 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:34661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:35121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:35121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=34223
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:46:26.583010 30493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:46:26.584439 30493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:46:26.596668 30506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:26.596702 30509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:26.596692 30505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:46:26.597605 30493 server_base.cc:1034] running on GCE node
I20260810 18:46:26.598443 30493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:46:26.599941 30493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:46:26.601205 30493 hybrid_clock.cc:648] HybridClock initialized: now 1786387586601124 us; error 46 us; skew 500 ppm
I20260810 18:46:26.601567 30493 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:46:26 dist-test-slave-7kzw krb5kdc[28843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387586, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:46:26.604991 30493 init.cc:377] Logged in from keytab as kudu/127.4.228.254@KRBTEST.COM (short username kudu)
I20260810 18:46:26.606549 30493 webserver.cc:460] Webserver started at http://127.4.228.254:34223/ using document root <none> and password file <none>
I20260810 18:46:26.607342 30493 fs_manager.cc:359] Metadata directory not provided
I20260810 18:46:26.607487 30493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:46:26.613157 30493 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:46:26.616853 30516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:26.618232 30493 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260810 18:46:26.618438 30493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "4d47ce6d53164cc6b711addede1613ca"
format_stamp: "Formatted at 2026-08-10 18:45:40 on dist-test-slave-7kzw"
server_key: "e80ea1563709bcc7d4ff985586949483"
server_key_iv: "47a9842fb3eff52a620674c42de1c8fe"
server_key_version: "encryptionkey@0"
I20260810 18:46:26.619484 30493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:46:26.650825 30493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:46:26.655202 30493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:46:26.655529 30493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:46:26.675653 30493 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:35121
I20260810 18:46:26.675684 30597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:35121 every 8 connection(s)
I20260810 18:46:26.677254 30493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:46:26.682548 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 30493
I20260810 18:46:26.683437 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.193:39905
--local_ip_for_outbound_sockets=127.4.228.193
--tserver_master_addrs=127.4.228.254:35121
--webserver_port=43447
--webserver_interface=127.4.228.193
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:34661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:46:26.684712 30598 sys_catalog.cc:263] Verifying existing consensus state
I20260810 18:46:26.688112 30598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca: Bootstrap starting.
I20260810 18:46:26.713675 30598 log.cc:824] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca: Log is configured to *not* fsync() on all Append() calls
I20260810 18:46:26.737116 30598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:46:26.737860 30598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca: Bootstrap complete.
I20260810 18:46:26.751910 30598 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d47ce6d53164cc6b711addede1613ca" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35121 } }
I20260810 18:46:26.753332 30598 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d47ce6d53164cc6b711addede1613ca, State: Initialized, Role: FOLLOWER
I20260810 18:46:26.754136 30598 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d47ce6d53164cc6b711addede1613ca" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35121 } }
I20260810 18:46:26.754503 30598 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:46:26.754642 30598 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:46:26.754849 30598 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [term 2 FOLLOWER]: Advancing to term 3
I20260810 18:46:26.760696 30598 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d47ce6d53164cc6b711addede1613ca" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35121 } }
I20260810 18:46:26.761389 30598 leader_election.cc:302] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d47ce6d53164cc6b711addede1613ca; no voters:
I20260810 18:46:26.762209 30598 leader_election.cc:288] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 18:46:26.762373 30603 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [term 3 FOLLOWER]: Leader election won for term 3
I20260810 18:46:26.764040 30603 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [term 3 LEADER]: Becoming Leader. State: Replica: 4d47ce6d53164cc6b711addede1613ca, State: Running, Role: LEADER
I20260810 18:46:26.764592 30603 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d47ce6d53164cc6b711addede1613ca" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35121 } }
I20260810 18:46:26.764868 30598 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:46:26.767410 30605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "4d47ce6d53164cc6b711addede1613ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d47ce6d53164cc6b711addede1613ca" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35121 } } }
I20260810 18:46:26.767405 30606 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d47ce6d53164cc6b711addede1613ca. Latest consensus state: current_term: 3 leader_uuid: "4d47ce6d53164cc6b711addede1613ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d47ce6d53164cc6b711addede1613ca" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35121 } } }
I20260810 18:46:26.767866 30605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [sys.catalog]: This master's current role is: LEADER
I20260810 18:46:26.767970 30606 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca [sys.catalog]: This master's current role is: LEADER
I20260810 18:46:26.768919 30611 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:46:26.781109 30611 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:46:26.782766 30611 catalog_manager.cc:1259] Loaded cluster ID: f39befd62fef402e98e6aef83721c785
I20260810 18:46:26.782969 30611 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:46:26.787494 30611 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:46:26.788764 30611 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca: Loaded TSK: 0
I20260810 18:46:26.788920 30611 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4d47ce6d53164cc6b711addede1613ca: Loaded TSK: 1
F20260810 18:46:26.791064 30611 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f771923d9ad google::LogMessage::Fail() at ??:0
@ 0x7f771924088a google::LogMessage::SendToLog() at ??:0
@ 0x7f771923d4de google::LogMessage::Flush() at ??:0
@ 0x7f771923ea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f771cfa17d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f771cf9073e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f772ec6c1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f772ec6b783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f772ecf72a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f772ecf70c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f772ed415e9 std::function<>::operator()() at ??:0
@ 0x7f772ec6fcdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f772ec6dc36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f772ecf5610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f772ecf5439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f771be89345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f771be8c91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f771be8c799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f771be67eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f771f1416db start_thread at ??:0
@ 0x7f77180d761f clone at ??:0
W20260810 18:46:27.148733 30600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:46:27.149098 30600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:46:27.149277 30600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:46:27.157130 30600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:46:27.157299 30600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:46:27.157483 30600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.193
I20260810 18:46:27.167541 30600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:34661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.193:39905
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.228.193
--webserver_port=43447
--tserver_master_addrs=127.4.228.254:35121
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.193
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:46:27.168972 30600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:46:27.170435 30600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:46:27.184273 30674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:27.184351 30672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:27.184319 30678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:46:27.185527 30600 server_base.cc:1034] running on GCE node
I20260810 18:46:27.186218 30600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:46:27.187331 30600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:46:27.188558 30600 hybrid_clock.cc:648] HybridClock initialized: now 1786387587188489 us; error 62 us; skew 500 ppm
I20260810 18:46:27.188916 30600 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:46:27 dist-test-slave-7kzw krb5kdc[28843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387587, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:46:27.192518 30600 init.cc:377] Logged in from keytab as kudu/127.4.228.193@KRBTEST.COM (short username kudu)
I20260810 18:46:27.194244 30600 webserver.cc:460] Webserver started at http://127.4.228.193:43447/ using document root <none> and password file <none>
I20260810 18:46:27.195082 30600 fs_manager.cc:359] Metadata directory not provided
I20260810 18:46:27.195263 30600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:46:27.201202 30600 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:46:27.204391 30689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:27.205753 30600 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 18:46:27.205948 30600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/wal
uuid: "3864ce298e8744258f3cdc6c68f97864"
format_stamp: "Formatted at 2026-08-10 18:45:41 on dist-test-slave-7kzw"
server_key: "76cb203db86fc72e217f15ef5039d2a9"
server_key_iv: "d84ca856c058ce9d594105efeb53255e"
server_key_version: "encryptionkey@0"
I20260810 18:46:27.207042 30600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:46:27.228103 30600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:46:27.232717 30600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:46:27.233012 30600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:46:27.234091 30600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:46:27.235580 30600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:46:27.235728 30600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:27.235874 30600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:46:27.235968 30600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:27.273113 30600 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.193:39905
I20260810 18:46:27.273273 30864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.193:39905 every 8 connection(s)
I20260810 18:46:27.274679 30600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
I20260810 18:46:27.279883 30867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:27.277884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.193:49213)
0810 18:46:27.278723 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:27.278976 (+ 253us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":332,"threads_started":1}
W20260810 18:46:27.280640 30865 heartbeater.cc:644] Failed to heartbeat to 127.4.228.254:35121 (0 consecutive failures): Network error: Failed to ping master at 127.4.228.254:35121: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
I20260810 18:46:27.281490 30867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:27.281133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.193:58665)
0810 18:46:27.281302 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:27.281384 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260810 18:46:27.282655 30867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:27.282162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.193:60383)
0810 18:46:27.282341 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:27.282434 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260810 18:46:27.282976 30865 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 18:46:27.283874 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 30600
I20260810 18:46:27.284616 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.194:45331
--local_ip_for_outbound_sockets=127.4.228.194
--tserver_master_addrs=127.4.228.254:35121
--webserver_port=37951
--webserver_interface=127.4.228.194
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:34661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:46:27.634282 30868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:46:27.634639 30868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:46:27.634744 30868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:46:27.641888 30868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:46:27.641996 30868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:46:27.642114 30868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.194
I20260810 18:46:27.651803 30868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:34661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.194:45331
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.228.194
--webserver_port=37951
--tserver_master_addrs=127.4.228.254:35121
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.194
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:46:27.653035 30868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:46:27.654649 30868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:46:27.669427 30873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:27.669456 30876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:27.670254 30880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:46:27.671329 30868 server_base.cc:1034] running on GCE node
I20260810 18:46:27.671962 30868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:46:27.673012 30868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:46:27.674253 30868 hybrid_clock.cc:648] HybridClock initialized: now 1786387587674200 us; error 41 us; skew 500 ppm
I20260810 18:46:27.674645 30868 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:46:27 dist-test-slave-7kzw krb5kdc[28843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387587, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:46:27.677800 30868 init.cc:377] Logged in from keytab as kudu/127.4.228.194@KRBTEST.COM (short username kudu)
I20260810 18:46:27.679396 30868 webserver.cc:460] Webserver started at http://127.4.228.194:37951/ using document root <none> and password file <none>
I20260810 18:46:27.680109 30868 fs_manager.cc:359] Metadata directory not provided
I20260810 18:46:27.680272 30868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:46:27.685683 30868 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 18:46:27.688443 30894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:27.689754 30868 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:46:27.689939 30868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/wal
uuid: "c04280931825462796891c073e8074ac"
format_stamp: "Formatted at 2026-08-10 18:45:41 on dist-test-slave-7kzw"
server_key: "3633085410b20a64de2bd071215e3de3"
server_key_iv: "1e79c438679ed9c0df5d9d902c593543"
server_key_version: "encryptionkey@0"
I20260810 18:46:27.690974 30868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:46:27.733191 30868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:46:27.737735 30868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:46:27.738031 30868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:46:27.739162 30868 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:46:27.740603 30868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:46:27.740746 30868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:27.740878 30868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:46:27.740970 30868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:27.776212 30868 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.194:45331
I20260810 18:46:27.776268 31066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.194:45331 every 8 connection(s)
I20260810 18:46:27.777803 30868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
I20260810 18:46:27.783007 31070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:27.780985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.194:51661)
0810 18:46:27.781750 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:27.782005 (+ 255us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":284,"threads_started":1}
W20260810 18:46:27.783718 31067 heartbeater.cc:644] Failed to heartbeat to 127.4.228.254:35121 (0 consecutive failures): Network error: Failed to ping master at 127.4.228.254:35121: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
I20260810 18:46:27.784618 31070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:27.784297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.194:33629)
0810 18:46:27.784441 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:27.784512 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 18:46:27.785542 31070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:27.785228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.194:35893)
0810 18:46:27.785363 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:27.785435 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260810 18:46:27.785815 31067 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 18:46:27.786365 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 30868
I20260810 18:46:27.787043 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.195:40527
--local_ip_for_outbound_sockets=127.4.228.195
--tserver_master_addrs=127.4.228.254:35121
--webserver_port=42191
--webserver_interface=127.4.228.195
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:34661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:46:28.143276 31073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:46:28.143618 31073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:46:28.143805 31073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:46:28.150987 31073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:46:28.151139 31073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:46:28.151346 31073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.195
I20260810 18:46:28.161317 31073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:34661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.195:40527
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.228.195
--webserver_port=42191
--tserver_master_addrs=127.4.228.254:35121
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.195
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:46:28.162714 31073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:46:28.164054 31073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:46:28.177706 31089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:28.177815 31086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:28.177771 31085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:46:28.178982 31073 server_base.cc:1034] running on GCE node
I20260810 18:46:28.179666 31073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:46:28.180672 31073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:46:28.181872 31073 hybrid_clock.cc:648] HybridClock initialized: now 1786387588181813 us; error 61 us; skew 500 ppm
I20260810 18:46:28.182147 31073 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:46:28 dist-test-slave-7kzw krb5kdc[28843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387588, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:46:28.185352 31073 init.cc:377] Logged in from keytab as kudu/127.4.228.195@KRBTEST.COM (short username kudu)
I20260810 18:46:28.186961 31073 webserver.cc:460] Webserver started at http://127.4.228.195:42191/ using document root <none> and password file <none>
I20260810 18:46:28.187700 31073 fs_manager.cc:359] Metadata directory not provided
I20260810 18:46:28.187865 31073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:46:28.193210 31073 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 18:46:28.196118 31101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:28.197430 31073 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260810 18:46:28.197608 31073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/wal
uuid: "23521ea509214ca89745701374ff8eda"
format_stamp: "Formatted at 2026-08-10 18:45:42 on dist-test-slave-7kzw"
server_key: "8d48b38acf11af152cf35838bc180760"
server_key_iv: "912a5d8864142603f986f7f3dcf7d872"
server_key_version: "encryptionkey@0"
I20260810 18:46:28.198585 31073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:46:28.220068 31073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:46:28.224386 31073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:46:28.224730 31073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:46:28.225769 31073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:46:28.227253 31073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:46:28.227401 31073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:28.227550 31073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:46:28.227653 31073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:28.262596 31073 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.195:40527
I20260810 18:46:28.262694 31284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.195:40527 every 8 connection(s)
I20260810 18:46:28.264097 31073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
I20260810 18:46:28.269161 31290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:28.267236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.195:51419)
0810 18:46:28.268048 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:28.268312 (+ 264us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":317,"threads_started":1}
I20260810 18:46:28.269255 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 31073
W20260810 18:46:28.269944 31287 heartbeater.cc:644] Failed to heartbeat to 127.4.228.254:35121 (0 consecutive failures): Network error: Failed to ping master at 127.4.228.254:35121: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
I20260810 18:46:28.270905 31290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:28.270534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.195:41501)
0810 18:46:28.270694 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:28.270764 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 18:46:28.271905 31290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:28.271490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.195:59043)
0810 18:46:28.271698 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:28.271778 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
W20260810 18:46:28.272300 31287 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 18:46:28.284288 31292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:28.283499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.193:36455)
0810 18:46:28.283968 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:28.284067 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":206,"threads_started":1}
I20260810 18:46:28.787226 31293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:28.786339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.194:53949)
0810 18:46:28.786876 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:28.786989 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":315,"threads_started":1}
I20260810 18:46:29.274051 31294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:29.272995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.195:55221)
0810 18:46:29.273630 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:29.273757 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":309,"threads_started":1}
I20260810 18:46:29.286121 31297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:29.285267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.193:55211)
0810 18:46:29.285869 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:29.285952 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":369,"threads_started":1}
I20260810 18:46:29.788920 31299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:29.788140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.194:49637)
0810 18:46:29.788631 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:29.788743 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":258,"threads_started":1}
I20260810 18:46:30.275957 31300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:30.275188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.195:32969)
0810 18:46:30.275692 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:30.275782 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":247,"threads_started":1}
I20260810 18:46:30.287879 31301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:30.287145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.193:60709)
0810 18:46:30.287603 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:30.287708 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":254,"threads_started":1}
I20260810 18:46:30.790777 31302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:30.789922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.194:39875)
0810 18:46:30.790434 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:30.790583 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":251,"threads_started":1}
I20260810 18:46:31.277631 31303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:31.276906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.195:56293)
0810 18:46:31.277347 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:31.277429 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":223,"threads_started":1}
I20260810 18:46:31.289749 31304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:31.288816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.193:60711)
0810 18:46:31.289378 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:31.289501 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":300,"threads_started":1}
I20260810 18:46:31.792407 31306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:31.791702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.194:53255)
0810 18:46:31.792142 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:31.792229 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":221,"threads_started":1}
I20260810 18:46:32.279435 31307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:32.278680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.195:55565)
0810 18:46:32.279157 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:32.279238 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":246,"threads_started":1}
I20260810 18:46:32.291491 31308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:32.290733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.193:48123)
0810 18:46:32.291225 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:32.291310 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":238,"threads_started":1}
I20260810 18:46:32.794261 31309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:32.793387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.194:38439)
0810 18:46:32.793960 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:32.794051 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":335,"threads_started":1}
I20260810 18:46:33.281237 31311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:33.280434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.195:38721)
0810 18:46:33.280976 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:33.281059 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":330,"threads_started":1}
I20260810 18:46:33.293215 31315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:33.292412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.193:40239)
0810 18:46:33.292939 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:33.293037 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":272,"threads_started":1}
I20260810 18:46:33.796134 31316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:33.795354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.194:38117)
0810 18:46:33.795847 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:33.795950 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":217,"threads_started":1}
I20260810 18:46:34.282963 31317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:34.282206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.195:54115)
0810 18:46:34.282703 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:34.282798 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":292,"threads_started":1}
I20260810 18:46:34.295127 31318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:34.294212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.193:46037)
0810 18:46:34.294820 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:34.294905 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":312,"threads_started":1}
I20260810 18:46:34.798019 31319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:34.797155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.194:39357)
0810 18:46:34.797705 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:34.797793 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":237,"threads_started":1}
I20260810 18:46:35.284744 31320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:35.283916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.195:51609)
0810 18:46:35.284432 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:35.284525 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":223,"threads_started":1}
I20260810 18:46:35.296725 31322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:35.296022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.193:51435)
0810 18:46:35.296468 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:35.296552 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":236,"threads_started":1}
I20260810 18:46:35.801726 31325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:35.799489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.194:43831)
0810 18:46:35.801229 (+ 1740us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:35.801360 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1608,"thread_start_us":1500,"threads_started":1}
I20260810 18:46:36.286618 31327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:36.285821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.195:49251)
0810 18:46:36.286296 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:36.286379 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":229,"threads_started":1}
I20260810 18:46:36.298588 31328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:36.297628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.193:45009)
0810 18:46:36.298253 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:36.298340 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":348,"threads_started":1}
I20260810 18:46:36.803743 31331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:36.802885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.194:57717)
0810 18:46:36.803398 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:36.803510 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":237,"threads_started":1}
I20260810 18:46:37.288447 31332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:37.287609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.195:40147)
0810 18:46:37.288161 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:37.288254 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":266,"threads_started":1}
I20260810 18:46:37.300352 31333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:37.299531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.193:41161)
0810 18:46:37.300056 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:37.300161 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":265,"threads_started":1}
I20260810 18:46:37.805591 31334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:37.804823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.194:34813)
0810 18:46:37.805310 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:37.805398 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":272,"threads_started":1}
I20260810 18:46:38.290306 31335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:38.289580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.195:32951)
0810 18:46:38.290052 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:38.290138 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":253,"threads_started":1}
I20260810 18:46:38.302217 31336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:38.301329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.193:58045)
0810 18:46:38.301899 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:38.302001 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":290,"threads_started":1}
I20260810 18:46:38.807567 31337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:38.806762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.194:54607)
0810 18:46:38.807261 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:38.807374 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":256,"threads_started":1}
I20260810 18:46:39.292594 31338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:39.291668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.195:45213)
0810 18:46:39.292298 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:39.292400 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":285,"threads_started":1}
I20260810 18:46:39.304160 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:39.303314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.193:43495)
0810 18:46:39.303894 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:39.303984 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":353,"threads_started":1}
I20260810 18:46:39.809466 31340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:39.808758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.194:58775)
0810 18:46:39.809210 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:39.809296 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":213,"threads_started":1}
I20260810 18:46:40.294365 31341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:40.293561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.195:34407)
0810 18:46:40.294069 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:40.294165 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":271,"threads_started":1}
I20260810 18:46:40.305979 31342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:40.305102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.193:57183)
0810 18:46:40.305665 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:40.305773 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":286,"threads_started":1}
I20260810 18:46:40.811612 31343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:40.810625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.194:49555)
0810 18:46:40.811253 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:40.811390 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":360,"threads_started":1}
I20260810 18:46:41.296408 31345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:41.295455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.195:50859)
0810 18:46:41.296093 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:41.296211 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":341,"threads_started":1}
I20260810 18:46:41.307911 31346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:41.307045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.193:58771)
0810 18:46:41.307636 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:41.307721 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":336,"threads_started":1}
I20260810 18:46:41.813428 31347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:41.812625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.194:52283)
0810 18:46:41.813123 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:41.813219 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":243,"threads_started":1}
I20260810 18:46:42.298085 31350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:42.297321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.195:42959)
0810 18:46:42.297787 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:42.297879 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":235,"threads_started":1}
I20260810 18:46:42.309782 31352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:42.308996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.193:35945)
0810 18:46:42.309460 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:42.309550 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":252,"threads_started":1}
I20260810 18:46:42.815719 31353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:42.814838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.194:50307)
0810 18:46:42.815391 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:42.815514 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":303,"threads_started":1}
I20260810 18:46:43.275861 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 30600
I20260810 18:46:43.299883 31356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:43.277298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42778 (local address 127.4.228.193:39905)
0810 18:46:43.278344 (+ 1046us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:43.278359 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:43.278398 (+ 39us) server_negotiation.cc:407] Connection header received
0810 18:46:43.280028 (+ 1630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:43.280048 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:43.280486 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:43.280797 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:43.282296 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:43.283704 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:43.285041 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:43.285761 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:43.285906 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:43.292410 (+ 6504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:43.292420 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:43.292437 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:46:43.292479 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:43.296014 (+ 3535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:43.296836 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:43.296873 (+ 37us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:43.296885 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:43.297092 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:43.297544 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:43.297550 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:43.297554 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:43.298171 (+ 617us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:43.298524 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:43.299082 (+ 558us) server_negotiation.cc:299] Negotiation successful
0810 18:46:43.299570 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":879,"thread_start_us":744,"threads_started":1}
I20260810 18:46:43.301083 31357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:43.299330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35121 (local address 127.4.228.195:36361)
0810 18:46:43.300788 (+ 1458us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:43.300899 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.254:35121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1316,"thread_start_us":1190,"threads_started":1}
I20260810 18:46:43.301470 30858 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:46:43.399168 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 30600
I20260810 18:46:43.430531 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 30868
I20260810 18:46:43.449484 31360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:43.431615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35336 (local address 127.4.228.194:45331)
0810 18:46:43.432188 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:43.432202 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:43.432240 (+ 38us) server_negotiation.cc:407] Connection header received
0810 18:46:43.433937 (+ 1697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:43.433949 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:43.434384 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:43.434871 (+ 487us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:43.436453 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:43.437743 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:43.439112 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:43.439783 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:43.439989 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:43.442881 (+ 2892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:43.442889 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:43.442905 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:46:43.442945 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:43.445828 (+ 2883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:43.446695 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:43.446715 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:43.446723 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:43.446900 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:43.447354 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:43.447359 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:43.447361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:43.447902 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:43.448236 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:43.448752 (+ 516us) server_negotiation.cc:299] Negotiation successful
0810 18:46:43.449186 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":271,"threads_started":1}
I20260810 18:46:43.451094 31061 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:46:43.540592 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 30868
I20260810 18:46:43.570293 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 31073
I20260810 18:46:43.589459 31362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:43.571515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52894 (local address 127.4.228.195:40527)
0810 18:46:43.572165 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:43.572181 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:43.572226 (+ 45us) server_negotiation.cc:407] Connection header received
0810 18:46:43.573903 (+ 1677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:43.573915 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:43.574356 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:43.574849 (+ 493us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:43.576439 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:43.577662 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:43.579078 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:43.579755 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:43.579974 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:43.582850 (+ 2876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:43.582859 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:43.582875 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:46:43.582917 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:43.585912 (+ 2995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:43.586684 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:43.586699 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:43.586705 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:43.586880 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:43.587333 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:43.587337 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:43.587339 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:43.587894 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:43.588241 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:43.588713 (+ 472us) server_negotiation.cc:299] Negotiation successful
0810 18:46:43.589133 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":309,"threads_started":1}
I20260810 18:46:43.590909 31279 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:46:43.678387 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 31073
2026-08-10T18:46:43Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65462 ms)
[----------] 30 tests from SecurityITest (335667 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:46:43 dist-test-slave-7kzw krb5kdc[31367](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:46:43 dist-test-slave-7kzw krb5kdc[31367](info): set up 2 sockets
Aug 10 18:46:43 dist-test-slave-7kzw krb5kdc[31367](info): commencing operation
krb5kdc: starting...
W20260810 18:46:45.030015 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.298s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:46:45 dist-test-slave-7kzw krb5kdc[31367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:46:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:46:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.228.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
I20260810 18:46:45.358551 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:38861
--webserver_interface=127.4.228.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:45247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:38861
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:46:45.722070 31386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:46:45.722433 31386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:46:45.722617 31386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:46:45.729904 31386 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:46:45.730057 31386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:46:45.730173 31386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:46:45.730276 31386 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:46:45.730356 31386 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:46:45.740473 31386 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:45247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:38861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:38861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:46:45.741890 31386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:46:45.743340 31386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:46:45.756158 31393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:45.756189 31394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:45.756124 31396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:46:45.757041 31386 server_base.cc:1034] running on GCE node
I20260810 18:46:45.758226 31386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:46:45.759800 31386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:46:45.761052 31386 hybrid_clock.cc:648] HybridClock initialized: now 1786387605761014 us; error 57 us; skew 500 ppm
I20260810 18:46:45.761451 31386 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:46:45 dist-test-slave-7kzw krb5kdc[31367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387605, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:46:45.764710 31386 init.cc:377] Logged in from keytab as kudu/127.4.228.254@KRBTEST.COM (short username kudu)
I20260810 18:46:45.766332 31386 webserver.cc:460] Webserver started at http://127.4.228.254:37433/ using document root <none> and password file <none>
I20260810 18:46:45.767104 31386 fs_manager.cc:359] Metadata directory not provided
I20260810 18:46:45.767263 31386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:46:45.767611 31386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:46:45.770758 31386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "509d883077e848e1b284c451d2efa6fc"
format_stamp: "Formatted at 2026-08-10 18:46:45 on dist-test-slave-7kzw"
server_key: "d456c1416c19d2321991a61ac182bb50"
server_key_iv: "f1d15fb4a2fe3a88c4506883be4b1989"
server_key_version: "encryptionkey@0"
I20260810 18:46:45.771554 31386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "509d883077e848e1b284c451d2efa6fc"
format_stamp: "Formatted at 2026-08-10 18:46:45 on dist-test-slave-7kzw"
server_key: "d456c1416c19d2321991a61ac182bb50"
server_key_iv: "f1d15fb4a2fe3a88c4506883be4b1989"
server_key_version: "encryptionkey@0"
I20260810 18:46:45.776976 31386 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260810 18:46:45.780763 31403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:45.782168 31386 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:46:45.782384 31386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "509d883077e848e1b284c451d2efa6fc"
format_stamp: "Formatted at 2026-08-10 18:46:45 on dist-test-slave-7kzw"
server_key: "d456c1416c19d2321991a61ac182bb50"
server_key_iv: "f1d15fb4a2fe3a88c4506883be4b1989"
server_key_version: "encryptionkey@0"
I20260810 18:46:45.782647 31386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:46:45.801222 31386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:46:45.806095 31386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:46:45.806428 31386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:46:45.831166 31386 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:38861
I20260810 18:46:45.831175 31490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:38861 every 8 connection(s)
I20260810 18:46:45.832648 31386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:46:45.836796 31491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:46:45.840516 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 31386
I20260810 18:46:45.840842 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:46:45.841641 5011 external_mini_cluster.cc:1428] Setting key fe7ceb6b4633f81833bb8c30eba8917a
I20260810 18:46:45.848570 31491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 509d883077e848e1b284c451d2efa6fc: Bootstrap starting.
I20260810 18:46:45.851933 31491 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 509d883077e848e1b284c451d2efa6fc: Neither blocks nor log segments found. Creating new log.
I20260810 18:46:45.852978 31491 log.cc:824] T 00000000000000000000000000000000 P 509d883077e848e1b284c451d2efa6fc: Log is configured to *not* fsync() on all Append() calls
I20260810 18:46:45.856303 31491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 509d883077e848e1b284c451d2efa6fc: No bootstrap required, opened a new log
Aug 10 18:46:45 dist-test-slave-7kzw krb5kdc[31367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:46:45.868299 31491 raft_consensus.cc:348] T 00000000000000000000000000000000 P 509d883077e848e1b284c451d2efa6fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "509d883077e848e1b284c451d2efa6fc" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 38861 } }
I20260810 18:46:45.868923 31491 raft_consensus.cc:374] T 00000000000000000000000000000000 P 509d883077e848e1b284c451d2efa6fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:46:45.869055 31491 raft_consensus.cc:729] T 00000000000000000000000000000000 P 509d883077e848e1b284c451d2efa6fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 509d883077e848e1b284c451d2efa6fc, State: Initialized, Role: FOLLOWER
I20260810 18:46:45.869547 31494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:45.843962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46322 (local address 127.4.228.254:38861)
0810 18:46:45.844860 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:45.844874 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:45.844920 (+ 46us) server_negotiation.cc:407] Connection header received
0810 18:46:45.847350 (+ 2430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:45.847378 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:45.847813 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:45.848244 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:45.849950 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:45.851499 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:45.853078 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:45.854281 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:45.854537 (+ 256us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:45.859173 (+ 4636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:45.859185 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:45.859208 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:46:45.859272 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:45.864014 (+ 4742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:45.864883 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:45.864894 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:45.864904 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:45.865180 (+ 276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:45.865586 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:45.865591 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:45.865596 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:45.866551 (+ 955us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:45.866881 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:45.867344 (+ 463us) server_negotiation.cc:299] Negotiation successful
0810 18:46:45.867926 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":396,"threads_started":1}
I20260810 18:46:45.869921 31491 consensus_queue.cc:260] T 00000000000000000000000000000000 P 509d883077e848e1b284c451d2efa6fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "509d883077e848e1b284c451d2efa6fc" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 38861 } }
I20260810 18:46:45.870234 31491 raft_consensus.cc:388] T 00000000000000000000000000000000 P 509d883077e848e1b284c451d2efa6fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:46:45.870419 31491 raft_consensus.cc:482] T 00000000000000000000000000000000 P 509d883077e848e1b284c451d2efa6fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:46:45.870702 31491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 509d883077e848e1b284c451d2efa6fc [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:46:45.873351 31491 raft_consensus.cc:504] T 00000000000000000000000000000000 P 509d883077e848e1b284c451d2efa6fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "509d883077e848e1b284c451d2efa6fc" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 38861 } }
I20260810 18:46:45.873912 31491 leader_election.cc:302] T 00000000000000000000000000000000 P 509d883077e848e1b284c451d2efa6fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 509d883077e848e1b284c451d2efa6fc; no voters:
I20260810 18:46:45.874645 31491 leader_election.cc:288] T 00000000000000000000000000000000 P 509d883077e848e1b284c451d2efa6fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:46:45.874788 31498 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 509d883077e848e1b284c451d2efa6fc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:46:45.875532 31498 raft_consensus.cc:686] T 00000000000000000000000000000000 P 509d883077e848e1b284c451d2efa6fc [term 1 LEADER]: Becoming Leader. State: Replica: 509d883077e848e1b284c451d2efa6fc, State: Running, Role: LEADER
I20260810 18:46:45.876134 31498 consensus_queue.cc:237] T 00000000000000000000000000000000 P 509d883077e848e1b284c451d2efa6fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "509d883077e848e1b284c451d2efa6fc" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 38861 } }
I20260810 18:46:45.876974 31491 sys_catalog.cc:564] T 00000000000000000000000000000000 P 509d883077e848e1b284c451d2efa6fc [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:46:45.879901 31499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 509d883077e848e1b284c451d2efa6fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "509d883077e848e1b284c451d2efa6fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "509d883077e848e1b284c451d2efa6fc" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 38861 } } }
I20260810 18:46:45.880333 31499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 509d883077e848e1b284c451d2efa6fc [sys.catalog]: This master's current role is: LEADER
I20260810 18:46:45.880788 31500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 509d883077e848e1b284c451d2efa6fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 509d883077e848e1b284c451d2efa6fc. Latest consensus state: current_term: 1 leader_uuid: "509d883077e848e1b284c451d2efa6fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "509d883077e848e1b284c451d2efa6fc" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 38861 } } }
I20260810 18:46:45.881075 31500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 509d883077e848e1b284c451d2efa6fc [sys.catalog]: This master's current role is: LEADER
I20260810 18:46:45.882182 31510 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:46:45.887435 31510 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:46:45.896729 31510 catalog_manager.cc:1347] Generated new cluster ID: af61e1d79eb0492cae0f0d6636913d43
I20260810 18:46:45.896878 31510 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:46:45.931056 31510 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:46:45.932617 31510 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:46:45.944097 31510 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 509d883077e848e1b284c451d2efa6fc: Generated new TSK 0
I20260810 18:46:45.945062 31510 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.228.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab.
Entry for principal kudu/127.4.228.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab.
I20260810 18:46:46.032645 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.193:0
--local_ip_for_outbound_sockets=127.4.228.193
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:38861
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:45247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:46:46.386948 31524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:46:46.387235 31524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:46:46.387460 31524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:46:46.394809 31524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:46:46.394913 31524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:46:46.395128 31524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.193
I20260810 18:46:46.405182 31524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:45247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:38861
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.193
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:46:46.406566 31524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:46:46.408005 31524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:46:46.421924 31531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:46.421922 31532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:46.422063 31535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:46:46.422545 31524 server_base.cc:1034] running on GCE node
I20260810 18:46:46.423334 31524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:46:46.424350 31524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:46:46.425585 31524 hybrid_clock.cc:648] HybridClock initialized: now 1786387606425531 us; error 43 us; skew 500 ppm
I20260810 18:46:46.425963 31524 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:46:46 dist-test-slave-7kzw krb5kdc[31367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387606, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:46:46.429471 31524 init.cc:377] Logged in from keytab as kudu/127.4.228.193@KRBTEST.COM (short username kudu)
I20260810 18:46:46.431209 31524 webserver.cc:460] Webserver started at http://127.4.228.193:35559/ using document root <none> and password file <none>
I20260810 18:46:46.431891 31524 fs_manager.cc:359] Metadata directory not provided
I20260810 18:46:46.431999 31524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:46:46.432287 31524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:46:46.435408 31524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-0/data/instance:
uuid: "27920ccf084247bfb6f2c859de429989"
format_stamp: "Formatted at 2026-08-10 18:46:46 on dist-test-slave-7kzw"
server_key: "eec051a7941b6f774deeee5fbb4e4bda"
server_key_iv: "a7689a1323b4cdbff6e3205376a8601f"
server_key_version: "encryptionkey@0"
I20260810 18:46:46.436183 31524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance:
uuid: "27920ccf084247bfb6f2c859de429989"
format_stamp: "Formatted at 2026-08-10 18:46:46 on dist-test-slave-7kzw"
server_key: "eec051a7941b6f774deeee5fbb4e4bda"
server_key_iv: "a7689a1323b4cdbff6e3205376a8601f"
server_key_version: "encryptionkey@0"
I20260810 18:46:46.441558 31524 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 18:46:46.445504 31547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:46.447105 31524 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:46:46.447268 31524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-0/wal
uuid: "27920ccf084247bfb6f2c859de429989"
format_stamp: "Formatted at 2026-08-10 18:46:46 on dist-test-slave-7kzw"
server_key: "eec051a7941b6f774deeee5fbb4e4bda"
server_key_iv: "a7689a1323b4cdbff6e3205376a8601f"
server_key_version: "encryptionkey@0"
I20260810 18:46:46.447460 31524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:46:46.463752 31524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:46:46.468187 31524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:46:46.468436 31524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:46:46.469406 31524 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:46:46.470844 31524 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:46:46.470922 31524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:46.471012 31524 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:46:46.471057 31524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:46.507095 31524 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.193:33471
I20260810 18:46:46.507151 31726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.193:33471 every 8 connection(s)
I20260810 18:46:46.508550 31524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
I20260810 18:46:46.514758 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 31524
I20260810 18:46:46.515053 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance
I20260810 18:46:46.515969 5011 external_mini_cluster.cc:1428] Setting key c4ea7b8dbe31455d67c4c475916461f0
Aug 10 18:46:46 dist-test-slave-7kzw krb5kdc[31367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387606, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:46:46.533588 31729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:46.511443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:59945 (local address 127.4.228.254:38861)
0810 18:46:46.511942 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:46.511947 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:46.513214 (+ 1267us) server_negotiation.cc:407] Connection header received
0810 18:46:46.516174 (+ 2960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:46.516180 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:46.516280 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:46.516409 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:46.518503 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:46.519581 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:46.521203 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:46.521918 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:46.522064 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:46.525852 (+ 3788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:46.525862 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:46.525869 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:46:46.525916 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:46.529866 (+ 3950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:46.530910 (+ 1044us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:46.530918 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:46.530923 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:46.531088 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:46.531610 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:46.531617 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:46.531622 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:46.532009 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:46.532182 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:46.533122 (+ 940us) server_negotiation.cc:299] Negotiation successful
0810 18:46:46.533311 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":251,"threads_started":1}
I20260810 18:46:46.535250 31730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:46.512029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:38861 (local address 127.4.228.193:59945)
0810 18:46:46.512986 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:46.513049 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:46.515768 (+ 2719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:46.516656 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:46.516670 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:46.517419 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:46:46.518258 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:46.518288 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:46.519781 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:46.519785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:46.521034 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:46.521047 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:46.522161 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:46.522166 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:46.522369 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:46.523175 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:46:46.523199 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:46:46.525585 (+ 2386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:46:46.530029 (+ 4444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:46:46.530038 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:46:46.530051 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:46:46.530772 (+ 721us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:46:46.531222 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:46:46.531228 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:46:46.531234 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:46:46.531482 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:46:46.532143 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:46:46.532154 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:46:46.532672 (+ 518us) client_negotiation.cc:769] Sending connection context
0810 18:46:46.533019 (+ 347us) client_negotiation.cc:240] Negotiation successful
0810 18:46:46.533629 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":316,"threads_started":1}
I20260810 18:46:46.536986 31727 heartbeater.cc:347] Connected to a master server at 127.4.228.254:38861
I20260810 18:46:46.537323 31727 heartbeater.cc:464] Registering TS with master...
I20260810 18:46:46.538131 31727 heartbeater.cc:511] Master 127.4.228.254:38861 requested a full tablet report, sending...
I20260810 18:46:46.540491 31432 ts_manager.cc:194] Registered new tserver with Master: 27920ccf084247bfb6f2c859de429989 (127.4.228.193:33471)
I20260810 18:46:46.541975 31432 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.193@KRBTEST.COM'} at 127.4.228.193:59945
WARNING: no policy specified for kudu/127.4.228.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab.
Entry for principal kudu/127.4.228.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab.
I20260810 18:46:46.589896 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.194:0
--local_ip_for_outbound_sockets=127.4.228.194
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:38861
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:45247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:46:46.946008 31735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:46:46.946345 31735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:46:46.946580 31735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:46:46.953944 31735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:46:46.954085 31735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:46:46.954272 31735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.194
I20260810 18:46:46.964076 31735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:45247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:38861
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.194
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:46:46.965430 31735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:46:46.966866 31735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:46:46.981441 31745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:46.981535 31750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:46.981467 31744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:46:46.982291 31735 server_base.cc:1034] running on GCE node
I20260810 18:46:46.983031 31735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:46:46.984055 31735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:46:46.985316 31735 hybrid_clock.cc:648] HybridClock initialized: now 1786387606985267 us; error 47 us; skew 500 ppm
I20260810 18:46:46.985677 31735 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:46:46 dist-test-slave-7kzw krb5kdc[31367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387606, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:46:46.989321 31735 init.cc:377] Logged in from keytab as kudu/127.4.228.194@KRBTEST.COM (short username kudu)
I20260810 18:46:46.991012 31735 webserver.cc:460] Webserver started at http://127.4.228.194:41721/ using document root <none> and password file <none>
I20260810 18:46:46.991772 31735 fs_manager.cc:359] Metadata directory not provided
I20260810 18:46:46.991925 31735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:46:46.992268 31735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:46:46.995388 31735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-1/data/instance:
uuid: "13f62e9bd8ef4060a976327db93aca7d"
format_stamp: "Formatted at 2026-08-10 18:46:46 on dist-test-slave-7kzw"
server_key: "26d01ac29d17ad9b2004fc4cd1b33612"
server_key_iv: "ee6af6c9b5e5fb96d88611b35ad25063"
server_key_version: "encryptionkey@0"
I20260810 18:46:46.996240 31735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance:
uuid: "13f62e9bd8ef4060a976327db93aca7d"
format_stamp: "Formatted at 2026-08-10 18:46:46 on dist-test-slave-7kzw"
server_key: "26d01ac29d17ad9b2004fc4cd1b33612"
server_key_iv: "ee6af6c9b5e5fb96d88611b35ad25063"
server_key_version: "encryptionkey@0"
I20260810 18:46:47.001752 31735 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 18:46:47.005760 31762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:47.007557 31735 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 18:46:47.007798 31735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-1/wal
uuid: "13f62e9bd8ef4060a976327db93aca7d"
format_stamp: "Formatted at 2026-08-10 18:46:46 on dist-test-slave-7kzw"
server_key: "26d01ac29d17ad9b2004fc4cd1b33612"
server_key_iv: "ee6af6c9b5e5fb96d88611b35ad25063"
server_key_version: "encryptionkey@0"
I20260810 18:46:47.008060 31735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:46:47.026757 31735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:46:47.031692 31735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:46:47.032127 31735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:46:47.033223 31735 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:46:47.034830 31735 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:46:47.034940 31735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:47.035120 31735 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:46:47.035195 31735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:47.072463 31735 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.194:43867
I20260810 18:46:47.072501 31926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.194:43867 every 8 connection(s)
I20260810 18:46:47.073956 31735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
I20260810 18:46:47.083277 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 31735
I20260810 18:46:47.083470 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance
I20260810 18:46:47.083925 5011 external_mini_cluster.cc:1428] Setting key 0cfa30e8b73d87b10a2ed666fb991c38
Aug 10 18:46:47 dist-test-slave-7kzw krb5kdc[31367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387606, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:46:47.095187 31931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:47.076712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:55413 (local address 127.4.228.254:38861)
0810 18:46:47.077202 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:47.077207 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:47.078244 (+ 1037us) server_negotiation.cc:407] Connection header received
0810 18:46:47.080180 (+ 1936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:47.080185 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:47.080280 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:47.080401 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:47.081912 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:47.082710 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:47.084436 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:47.085020 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:47.085158 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:47.089113 (+ 3955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:47.089119 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:47.089123 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:46:47.089158 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:47.091957 (+ 2799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:47.092709 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:47.092717 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:47.092722 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:47.092870 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:47.093360 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:47.093366 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:47.093369 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:47.093647 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:47.093833 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:47.094553 (+ 720us) server_negotiation.cc:299] Negotiation successful
0810 18:46:47.094796 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":264,"threads_started":1}
I20260810 18:46:47.096199 31932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:47.077285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:38861 (local address 127.4.228.194:55413)
0810 18:46:47.078082 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:47.078122 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:47.079898 (+ 1776us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:47.080652 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:47.080660 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:47.081159 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:46:47.081719 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:47.081736 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:47.082880 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:47.082884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:47.084271 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:47.084284 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:47.085249 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:47.085267 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:47.085614 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:47.086427 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:46:47.086497 (+ 70us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:46:47.088864 (+ 2367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:46:47.092116 (+ 3252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:46:47.092125 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:46:47.092142 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:46:47.092579 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:46:47.092991 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:46:47.092997 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:46:47.093000 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:46:47.093235 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:46:47.093785 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:46:47.093794 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:46:47.094156 (+ 362us) client_negotiation.cc:769] Sending connection context
0810 18:46:47.094446 (+ 290us) client_negotiation.cc:240] Negotiation successful
0810 18:46:47.095016 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":325,"threads_started":1}
I20260810 18:46:47.097774 31927 heartbeater.cc:347] Connected to a master server at 127.4.228.254:38861
I20260810 18:46:47.098104 31927 heartbeater.cc:464] Registering TS with master...
I20260810 18:46:47.098935 31927 heartbeater.cc:511] Master 127.4.228.254:38861 requested a full tablet report, sending...
I20260810 18:46:47.100489 31432 ts_manager.cc:194] Registered new tserver with Master: 13f62e9bd8ef4060a976327db93aca7d (127.4.228.194:43867)
I20260810 18:46:47.101456 31432 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.194@KRBTEST.COM'} at 127.4.228.194:55413
WARNING: no policy specified for kudu/127.4.228.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab.
Entry for principal kudu/127.4.228.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab.
I20260810 18:46:47.153497 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.195:0
--local_ip_for_outbound_sockets=127.4.228.195
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:38861
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:45247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:46:47.510231 31936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:46:47.510672 31936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:46:47.510854 31936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:46:47.519928 31936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:46:47.520078 31936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:46:47.520251 31936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.195
I20260810 18:46:47.530256 31936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:45247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:38861
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.195
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:46:47.531780 31936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:46:47.533193 31936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 18:46:47.545038 31727 heartbeater.cc:503] Master 127.4.228.254:38861 was elected leader, sending a full tablet report...
W20260810 18:46:47.546823 31942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:47.546938 31943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:47.547853 31945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:46:47.548341 31936 server_base.cc:1034] running on GCE node
I20260810 18:46:47.548992 31936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:46:47.549993 31936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:46:47.551270 31936 hybrid_clock.cc:648] HybridClock initialized: now 1786387607551186 us; error 45 us; skew 500 ppm
I20260810 18:46:47.551707 31936 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:46:47 dist-test-slave-7kzw krb5kdc[31367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387607, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:46:47.555325 31936 init.cc:377] Logged in from keytab as kudu/127.4.228.195@KRBTEST.COM (short username kudu)
I20260810 18:46:47.556840 31936 webserver.cc:460] Webserver started at http://127.4.228.195:41445/ using document root <none> and password file <none>
I20260810 18:46:47.557576 31936 fs_manager.cc:359] Metadata directory not provided
I20260810 18:46:47.557730 31936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:46:47.558063 31936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:46:47.561138 31936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-2/data/instance:
uuid: "c33d68f5cc3440a3a1b9b0f2b5c6c47e"
format_stamp: "Formatted at 2026-08-10 18:46:47 on dist-test-slave-7kzw"
server_key: "4f15000132af518ad557e9de0b89e01b"
server_key_iv: "bad5c6df5e26bcb551054ac29a58f01f"
server_key_version: "encryptionkey@0"
I20260810 18:46:47.561973 31936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance:
uuid: "c33d68f5cc3440a3a1b9b0f2b5c6c47e"
format_stamp: "Formatted at 2026-08-10 18:46:47 on dist-test-slave-7kzw"
server_key: "4f15000132af518ad557e9de0b89e01b"
server_key_iv: "bad5c6df5e26bcb551054ac29a58f01f"
server_key_version: "encryptionkey@0"
I20260810 18:46:47.567315 31936 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 18:46:47.570976 31958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:47.572412 31936 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:46:47.572623 31936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-2/wal
uuid: "c33d68f5cc3440a3a1b9b0f2b5c6c47e"
format_stamp: "Formatted at 2026-08-10 18:46:47 on dist-test-slave-7kzw"
server_key: "4f15000132af518ad557e9de0b89e01b"
server_key_iv: "bad5c6df5e26bcb551054ac29a58f01f"
server_key_version: "encryptionkey@0"
I20260810 18:46:47.572844 31936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:46:47.587105 31936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:46:47.591457 31936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:46:47.591756 31936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:46:47.592711 31936 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:46:47.594065 31936 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:46:47.594169 31936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:47.594328 31936 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:46:47.594404 31936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:47.631016 31936 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.195:36373
I20260810 18:46:47.631045 32148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.195:36373 every 8 connection(s)
I20260810 18:46:47.632390 31936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
I20260810 18:46:47.636624 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 31936
I20260810 18:46:47.636965 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance
I20260810 18:46:47.637482 5011 external_mini_cluster.cc:1428] Setting key 653f2a2b18857ba0ff7dc3f421a3ca31
Aug 10 18:46:47 dist-test-slave-7kzw krb5kdc[31367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387607, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:46:47.653379 32153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:47.635103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:50517 (local address 127.4.228.254:38861)
0810 18:46:47.635651 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:47.635656 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:47.638778 (+ 3122us) server_negotiation.cc:407] Connection header received
0810 18:46:47.638858 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:47.638863 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:47.638952 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:47.639069 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:47.640969 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:47.641779 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:47.643379 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:47.643806 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:47.643906 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:47.647326 (+ 3420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:47.647331 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:47.647334 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:46:47.647367 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:47.649837 (+ 2470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:47.650605 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:47.650609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:47.650612 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:47.650751 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:47.651227 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:47.651231 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:47.651233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:47.651567 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:47.651746 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:47.652703 (+ 957us) server_negotiation.cc:299] Negotiation successful
0810 18:46:47.652948 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":297,"threads_started":1}
I20260810 18:46:47.654435 32155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:47.635523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:38861 (local address 127.4.228.195:50517)
0810 18:46:47.636415 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:47.636513 (+ 98us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:47.638572 (+ 2059us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:47.639372 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:47.639384 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:47.640090 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:46:47.640778 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:47.640799 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:47.641933 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:47.641937 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:47.643243 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:47.643255 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:47.644010 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:47.644015 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:47.644217 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:47.644909 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:46:47.644932 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:46:47.647118 (+ 2186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:46:47.649974 (+ 2856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:46:47.649981 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:46:47.649994 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:46:47.650428 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:46:47.650868 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:46:47.650875 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:46:47.650881 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:46:47.651083 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:46:47.651754 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:46:47.651764 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:46:47.652238 (+ 474us) client_negotiation.cc:769] Sending connection context
0810 18:46:47.652575 (+ 337us) client_negotiation.cc:240] Negotiation successful
0810 18:46:47.653152 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":340,"threads_started":1}
I20260810 18:46:47.656024 32149 heartbeater.cc:347] Connected to a master server at 127.4.228.254:38861
I20260810 18:46:47.656383 32149 heartbeater.cc:464] Registering TS with master...
I20260810 18:46:47.657217 32149 heartbeater.cc:511] Master 127.4.228.254:38861 requested a full tablet report, sending...
I20260810 18:46:47.658794 31432 ts_manager.cc:194] Registered new tserver with Master: c33d68f5cc3440a3a1b9b0f2b5c6c47e (127.4.228.195:36373)
I20260810 18:46:47.659668 31432 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.195@KRBTEST.COM'} at 127.4.228.195:50517
I20260810 18:46:47.666786 5011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:46:47.687104 32153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:47.673091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46326 (local address 127.4.228.254:38861)
0810 18:46:47.673265 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:47.673292 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:47.673777 (+ 485us) server_negotiation.cc:407] Connection header received
0810 18:46:47.673999 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:47.674004 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:47.674100 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:47.674210 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:47.675849 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:47.676695 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:47.678164 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:47.678749 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:47.678886 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:47.681478 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:47.681484 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:47.681489 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:46:47.681573 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:47.684232 (+ 2659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:47.684935 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:47.684941 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:47.684944 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:47.685079 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:47.685472 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:47.685480 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:47.685484 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:47.685829 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:47.686089 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:47.686537 (+ 448us) server_negotiation.cc:299] Negotiation successful
0810 18:46:47.686781 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 18:46:47.691469 31432 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46326:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:46:47.698626 31432 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:46:47.735438 32173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:47.712599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48362 (local address 127.4.228.195:36373)
0810 18:46:47.726327 (+ 13728us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:47.726339 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:47.726378 (+ 39us) server_negotiation.cc:407] Connection header received
0810 18:46:47.726515 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:47.726523 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:47.726802 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:47.727023 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:46:47.727906 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:47.729602 (+ 1696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:47.732526 (+ 2924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:47.734007 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:47.734191 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:47.734477 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:47.734878 (+ 401us) server_negotiation.cc:299] Negotiation successful
0810 18:46:47.735059 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13481,"thread_start_us":13291,"threads_started":1}
I20260810 18:46:47.735679 32174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:47.712391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:36373 (local address 127.0.0.1:48362)
0810 18:46:47.723236 (+ 10845us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:47.723316 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:47.723562 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:47.727089 (+ 3527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:47.727097 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:47.727131 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:46:47.727656 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:47.727675 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:47.729813 (+ 2138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:47.729821 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:47.732315 (+ 2494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:47.732332 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:47.734256 (+ 1924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:47.734264 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:47.734580 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:47.734649 (+ 69us) client_negotiation.cc:769] Sending connection context
0810 18:46:47.734825 (+ 176us) client_negotiation.cc:240] Negotiation successful
0810 18:46:47.735071 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10430,"thread_start_us":13925,"threads_started":1}
I20260810 18:46:47.737357 32172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:47.713193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:33471 (local address 127.0.0.1:43394)
0810 18:46:47.726663 (+ 13470us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:47.726685 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:47.726847 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:47.727854 (+ 1007us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:47.727859 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:47.727888 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:46:47.729716 (+ 1828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:47.729732 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:47.731904 (+ 2172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:47.731909 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:47.734305 (+ 2396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:47.734321 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:47.736579 (+ 2258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:47.736584 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:47.736712 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:47.736735 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 18:46:47.736877 (+ 142us) client_negotiation.cc:240] Negotiation successful
0810 18:46:47.737013 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13331,"thread_start_us":10624,"threads_started":1}
I20260810 18:46:47.738061 32175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:47.713774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43394 (local address 127.4.228.193:33471)
0810 18:46:47.726507 (+ 12733us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:47.726519 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:47.726774 (+ 255us) server_negotiation.cc:407] Connection header received
0810 18:46:47.727066 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:47.727076 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:47.727326 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:47.727544 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:46:47.729965 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:47.731723 (+ 1758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:47.734537 (+ 2814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:47.736036 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:47.736186 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:47.736435 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:47.736924 (+ 489us) server_negotiation.cc:299] Negotiation successful
0810 18:46:47.737740 (+ 816us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12487,"thread_start_us":12738,"threads_started":1}
I20260810 18:46:47.740232 32176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:47.727047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47168 (local address 127.4.228.194:43867)
0810 18:46:47.731162 (+ 4115us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:47.731171 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:47.731314 (+ 143us) server_negotiation.cc:407] Connection header received
0810 18:46:47.731539 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:47.731544 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:47.731699 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:47.731927 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:46:47.732952 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:47.734541 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:47.737184 (+ 2643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:47.738714 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:47.738874 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:47.739112 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:47.739661 (+ 549us) server_negotiation.cc:299] Negotiation successful
0810 18:46:47.739846 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3551,"thread_start_us":3428,"threads_started":1}
I20260810 18:46:47.740929 32177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:47.727527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:43867 (local address 127.0.0.1:47168)
0810 18:46:47.731175 (+ 3648us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:47.731198 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:47.731363 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:47.732198 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:47.732204 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:47.732229 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:46:47.732741 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:47.732754 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:47.734688 (+ 1934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:47.734693 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:47.736936 (+ 2243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:47.736955 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:47.739273 (+ 2318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:47.739278 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:47.739401 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:47.739426 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 18:46:47.739555 (+ 129us) client_negotiation.cc:240] Negotiation successful
0810 18:46:47.739692 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3524,"thread_start_us":3339,"threads_started":1}
I20260810 18:46:47.740624 32038 tablet_service.cc:1467] Processing CreateTablet for tablet a1077e54cb7141f89526fcb0f754ccae (DEFAULT_TABLE table=test-table [id=e82133475eaa4e49aca1dc7eca681218]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:46:47.742368 32038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1077e54cb7141f89526fcb0f754ccae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:46:47.741839 31630 tablet_service.cc:1467] Processing CreateTablet for tablet a1077e54cb7141f89526fcb0f754ccae (DEFAULT_TABLE table=test-table [id=e82133475eaa4e49aca1dc7eca681218]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:46:47.743328 31630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1077e54cb7141f89526fcb0f754ccae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:46:47.747268 31830 tablet_service.cc:1467] Processing CreateTablet for tablet a1077e54cb7141f89526fcb0f754ccae (DEFAULT_TABLE table=test-table [id=e82133475eaa4e49aca1dc7eca681218]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:46:47.748442 31830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1077e54cb7141f89526fcb0f754ccae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:46:47.758175 32180 tablet_bootstrap.cc:492] T a1077e54cb7141f89526fcb0f754ccae P 27920ccf084247bfb6f2c859de429989: Bootstrap starting.
I20260810 18:46:47.759027 32179 tablet_bootstrap.cc:492] T a1077e54cb7141f89526fcb0f754ccae P c33d68f5cc3440a3a1b9b0f2b5c6c47e: Bootstrap starting.
I20260810 18:46:47.763243 32181 tablet_bootstrap.cc:492] T a1077e54cb7141f89526fcb0f754ccae P 13f62e9bd8ef4060a976327db93aca7d: Bootstrap starting.
I20260810 18:46:47.763622 32180 tablet_bootstrap.cc:654] T a1077e54cb7141f89526fcb0f754ccae P 27920ccf084247bfb6f2c859de429989: Neither blocks nor log segments found. Creating new log.
I20260810 18:46:47.763878 32179 tablet_bootstrap.cc:654] T a1077e54cb7141f89526fcb0f754ccae P c33d68f5cc3440a3a1b9b0f2b5c6c47e: Neither blocks nor log segments found. Creating new log.
I20260810 18:46:47.765280 32179 log.cc:824] T a1077e54cb7141f89526fcb0f754ccae P c33d68f5cc3440a3a1b9b0f2b5c6c47e: Log is configured to *not* fsync() on all Append() calls
I20260810 18:46:47.766101 32180 log.cc:824] T a1077e54cb7141f89526fcb0f754ccae P 27920ccf084247bfb6f2c859de429989: Log is configured to *not* fsync() on all Append() calls
I20260810 18:46:47.767316 32181 tablet_bootstrap.cc:654] T a1077e54cb7141f89526fcb0f754ccae P 13f62e9bd8ef4060a976327db93aca7d: Neither blocks nor log segments found. Creating new log.
I20260810 18:46:47.768244 32179 tablet_bootstrap.cc:492] T a1077e54cb7141f89526fcb0f754ccae P c33d68f5cc3440a3a1b9b0f2b5c6c47e: No bootstrap required, opened a new log
I20260810 18:46:47.768529 32179 ts_tablet_manager.cc:1397] T a1077e54cb7141f89526fcb0f754ccae P c33d68f5cc3440a3a1b9b0f2b5c6c47e: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260810 18:46:47.768927 32181 log.cc:824] T a1077e54cb7141f89526fcb0f754ccae P 13f62e9bd8ef4060a976327db93aca7d: Log is configured to *not* fsync() on all Append() calls
I20260810 18:46:47.773442 32181 tablet_bootstrap.cc:492] T a1077e54cb7141f89526fcb0f754ccae P 13f62e9bd8ef4060a976327db93aca7d: No bootstrap required, opened a new log
I20260810 18:46:47.773810 32181 ts_tablet_manager.cc:1397] T a1077e54cb7141f89526fcb0f754ccae P 13f62e9bd8ef4060a976327db93aca7d: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.007s
I20260810 18:46:47.778174 32179 raft_consensus.cc:348] T a1077e54cb7141f89526fcb0f754ccae P c33d68f5cc3440a3a1b9b0f2b5c6c47e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c33d68f5cc3440a3a1b9b0f2b5c6c47e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36373 } } peers { permanent_uuid: "13f62e9bd8ef4060a976327db93aca7d" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43867 } } peers { permanent_uuid: "27920ccf084247bfb6f2c859de429989" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 33471 } }
I20260810 18:46:47.778594 32179 raft_consensus.cc:374] T a1077e54cb7141f89526fcb0f754ccae P c33d68f5cc3440a3a1b9b0f2b5c6c47e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:46:47.778708 32179 raft_consensus.cc:729] T a1077e54cb7141f89526fcb0f754ccae P c33d68f5cc3440a3a1b9b0f2b5c6c47e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c33d68f5cc3440a3a1b9b0f2b5c6c47e, State: Initialized, Role: FOLLOWER
I20260810 18:46:47.779484 32180 tablet_bootstrap.cc:492] T a1077e54cb7141f89526fcb0f754ccae P 27920ccf084247bfb6f2c859de429989: No bootstrap required, opened a new log
I20260810 18:46:47.779366 32179 consensus_queue.cc:260] T a1077e54cb7141f89526fcb0f754ccae P c33d68f5cc3440a3a1b9b0f2b5c6c47e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c33d68f5cc3440a3a1b9b0f2b5c6c47e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36373 } } peers { permanent_uuid: "13f62e9bd8ef4060a976327db93aca7d" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43867 } } peers { permanent_uuid: "27920ccf084247bfb6f2c859de429989" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 33471 } }
I20260810 18:46:47.779798 32180 ts_tablet_manager.cc:1397] T a1077e54cb7141f89526fcb0f754ccae P 27920ccf084247bfb6f2c859de429989: Time spent bootstrapping tablet: real 0.022s user 0.009s sys 0.008s
I20260810 18:46:47.780624 32149 heartbeater.cc:503] Master 127.4.228.254:38861 was elected leader, sending a full tablet report...
I20260810 18:46:47.781433 32179 ts_tablet_manager.cc:1428] T a1077e54cb7141f89526fcb0f754ccae P c33d68f5cc3440a3a1b9b0f2b5c6c47e: Time spent starting tablet: real 0.013s user 0.004s sys 0.009s
I20260810 18:46:47.787571 32181 raft_consensus.cc:348] T a1077e54cb7141f89526fcb0f754ccae P 13f62e9bd8ef4060a976327db93aca7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c33d68f5cc3440a3a1b9b0f2b5c6c47e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36373 } } peers { permanent_uuid: "13f62e9bd8ef4060a976327db93aca7d" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43867 } } peers { permanent_uuid: "27920ccf084247bfb6f2c859de429989" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 33471 } }
I20260810 18:46:47.788028 32181 raft_consensus.cc:374] T a1077e54cb7141f89526fcb0f754ccae P 13f62e9bd8ef4060a976327db93aca7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:46:47.788157 32181 raft_consensus.cc:729] T a1077e54cb7141f89526fcb0f754ccae P 13f62e9bd8ef4060a976327db93aca7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13f62e9bd8ef4060a976327db93aca7d, State: Initialized, Role: FOLLOWER
I20260810 18:46:47.788828 32181 consensus_queue.cc:260] T a1077e54cb7141f89526fcb0f754ccae P 13f62e9bd8ef4060a976327db93aca7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c33d68f5cc3440a3a1b9b0f2b5c6c47e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36373 } } peers { permanent_uuid: "13f62e9bd8ef4060a976327db93aca7d" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43867 } } peers { permanent_uuid: "27920ccf084247bfb6f2c859de429989" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 33471 } }
I20260810 18:46:47.789767 32180 raft_consensus.cc:348] T a1077e54cb7141f89526fcb0f754ccae P 27920ccf084247bfb6f2c859de429989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c33d68f5cc3440a3a1b9b0f2b5c6c47e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36373 } } peers { permanent_uuid: "13f62e9bd8ef4060a976327db93aca7d" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43867 } } peers { permanent_uuid: "27920ccf084247bfb6f2c859de429989" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 33471 } }
I20260810 18:46:47.790033 31927 heartbeater.cc:503] Master 127.4.228.254:38861 was elected leader, sending a full tablet report...
I20260810 18:46:47.790167 32180 raft_consensus.cc:374] T a1077e54cb7141f89526fcb0f754ccae P 27920ccf084247bfb6f2c859de429989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:46:47.790294 32180 raft_consensus.cc:729] T a1077e54cb7141f89526fcb0f754ccae P 27920ccf084247bfb6f2c859de429989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27920ccf084247bfb6f2c859de429989, State: Initialized, Role: FOLLOWER
I20260810 18:46:47.790757 32181 ts_tablet_manager.cc:1428] T a1077e54cb7141f89526fcb0f754ccae P 13f62e9bd8ef4060a976327db93aca7d: Time spent starting tablet: real 0.017s user 0.011s sys 0.005s
I20260810 18:46:47.790970 32180 consensus_queue.cc:260] T a1077e54cb7141f89526fcb0f754ccae P 27920ccf084247bfb6f2c859de429989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c33d68f5cc3440a3a1b9b0f2b5c6c47e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36373 } } peers { permanent_uuid: "13f62e9bd8ef4060a976327db93aca7d" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43867 } } peers { permanent_uuid: "27920ccf084247bfb6f2c859de429989" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 33471 } }
I20260810 18:46:47.793541 32180 ts_tablet_manager.cc:1428] T a1077e54cb7141f89526fcb0f754ccae P 27920ccf084247bfb6f2c859de429989: Time spent starting tablet: real 0.014s user 0.009s sys 0.003s
W20260810 18:46:47.826206 31928 tablet.cc:2369] T a1077e54cb7141f89526fcb0f754ccae P 13f62e9bd8ef4060a976327db93aca7d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:46:47.884034 32150 tablet.cc:2369] T a1077e54cb7141f89526fcb0f754ccae P c33d68f5cc3440a3a1b9b0f2b5c6c47e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:46:47.928701 32191 raft_consensus.cc:482] T a1077e54cb7141f89526fcb0f754ccae P 27920ccf084247bfb6f2c859de429989 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:46:47.929070 32191 raft_consensus.cc:504] T a1077e54cb7141f89526fcb0f754ccae P 27920ccf084247bfb6f2c859de429989 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c33d68f5cc3440a3a1b9b0f2b5c6c47e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36373 } } peers { permanent_uuid: "13f62e9bd8ef4060a976327db93aca7d" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43867 } } peers { permanent_uuid: "27920ccf084247bfb6f2c859de429989" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 33471 } }
I20260810 18:46:47.931120 32191 leader_election.cc:288] T a1077e54cb7141f89526fcb0f754ccae P 27920ccf084247bfb6f2c859de429989 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c33d68f5cc3440a3a1b9b0f2b5c6c47e (127.4.228.195:36373), 13f62e9bd8ef4060a976327db93aca7d (127.4.228.194:43867)
I20260810 18:46:47.938267 32173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:47.931735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:54747 (local address 127.4.228.195:36373)
0810 18:46:47.931956 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:47.931961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:47.932409 (+ 448us) server_negotiation.cc:407] Connection header received
0810 18:46:47.932657 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:47.932667 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:47.932759 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:47.932827 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:46:47.933609 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:47.934764 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:47.936556 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:47.937319 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:47.937516 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:47.937598 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:47.937879 (+ 281us) server_negotiation.cc:299] Negotiation successful
0810 18:46:47.938008 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 18:46:47.938413 32192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:47.931518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:36373 (local address 127.4.228.193:54747)
0810 18:46:47.932256 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:47.932278 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:47.932434 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:47.933032 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:47.933035 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:47.933057 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:46:47.933463 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:47.933472 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:47.934887 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:47.934890 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:47.936369 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:47.936382 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:47.937531 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:47.937537 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:47.937702 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:47.937727 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 18:46:47.937849 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 18:46:47.937975 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"spinlock_wait_cycles":50048,"thread_start_us":400,"threads_started":1}
I20260810 18:46:47.938570 32176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:47.932666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:55309 (local address 127.4.228.194:43867)
0810 18:46:47.932891 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:47.932897 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:47.933165 (+ 268us) server_negotiation.cc:407] Connection header received
0810 18:46:47.933451 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:47.933455 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:47.933515 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:47.933635 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:46:47.934112 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:47.935094 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:47.936828 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:47.937592 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:47.937788 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:47.937866 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:47.938142 (+ 276us) server_negotiation.cc:299] Negotiation successful
0810 18:46:47.938272 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 18:46:47.938951 32194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:47.932412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:43867 (local address 127.4.228.193:55309)
0810 18:46:47.933040 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:47.933055 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:47.933280 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:47.933657 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:47.933660 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:47.933680 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:46:47.933954 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:47.933960 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:47.935238 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:47.935244 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:47.936660 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:47.936673 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:47.937831 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:47.937837 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:47.937985 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:47.938008 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 18:46:47.938123 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 18:46:47.938261 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":292,"threads_started":1}
I20260810 18:46:47.939507 32070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1077e54cb7141f89526fcb0f754ccae" candidate_uuid: "27920ccf084247bfb6f2c859de429989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c33d68f5cc3440a3a1b9b0f2b5c6c47e" is_pre_election: true
I20260810 18:46:47.939898 31863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1077e54cb7141f89526fcb0f754ccae" candidate_uuid: "27920ccf084247bfb6f2c859de429989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13f62e9bd8ef4060a976327db93aca7d" is_pre_election: true
I20260810 18:46:47.940021 32070 raft_consensus.cc:2454] T a1077e54cb7141f89526fcb0f754ccae P c33d68f5cc3440a3a1b9b0f2b5c6c47e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27920ccf084247bfb6f2c859de429989 in term 0.
I20260810 18:46:47.940382 31863 raft_consensus.cc:2454] T a1077e54cb7141f89526fcb0f754ccae P 13f62e9bd8ef4060a976327db93aca7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27920ccf084247bfb6f2c859de429989 in term 0.
I20260810 18:46:47.940735 31552 leader_election.cc:302] T a1077e54cb7141f89526fcb0f754ccae P 27920ccf084247bfb6f2c859de429989 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27920ccf084247bfb6f2c859de429989, c33d68f5cc3440a3a1b9b0f2b5c6c47e; no voters:
I20260810 18:46:47.941171 32191 raft_consensus.cc:2781] T a1077e54cb7141f89526fcb0f754ccae P 27920ccf084247bfb6f2c859de429989 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:46:47.941358 32191 raft_consensus.cc:482] T a1077e54cb7141f89526fcb0f754ccae P 27920ccf084247bfb6f2c859de429989 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:46:47.941495 32191 raft_consensus.cc:3037] T a1077e54cb7141f89526fcb0f754ccae P 27920ccf084247bfb6f2c859de429989 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:46:47.943819 32191 raft_consensus.cc:504] T a1077e54cb7141f89526fcb0f754ccae P 27920ccf084247bfb6f2c859de429989 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c33d68f5cc3440a3a1b9b0f2b5c6c47e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36373 } } peers { permanent_uuid: "13f62e9bd8ef4060a976327db93aca7d" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43867 } } peers { permanent_uuid: "27920ccf084247bfb6f2c859de429989" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 33471 } }
I20260810 18:46:47.944410 32191 leader_election.cc:288] T a1077e54cb7141f89526fcb0f754ccae P 27920ccf084247bfb6f2c859de429989 [CANDIDATE]: Term 1 election: Requested vote from peers c33d68f5cc3440a3a1b9b0f2b5c6c47e (127.4.228.195:36373), 13f62e9bd8ef4060a976327db93aca7d (127.4.228.194:43867)
I20260810 18:46:47.944967 32070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1077e54cb7141f89526fcb0f754ccae" candidate_uuid: "27920ccf084247bfb6f2c859de429989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c33d68f5cc3440a3a1b9b0f2b5c6c47e"
I20260810 18:46:47.944990 31863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1077e54cb7141f89526fcb0f754ccae" candidate_uuid: "27920ccf084247bfb6f2c859de429989" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13f62e9bd8ef4060a976327db93aca7d"
I20260810 18:46:47.945168 32070 raft_consensus.cc:3037] T a1077e54cb7141f89526fcb0f754ccae P c33d68f5cc3440a3a1b9b0f2b5c6c47e [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:46:47.945184 31863 raft_consensus.cc:3037] T a1077e54cb7141f89526fcb0f754ccae P 13f62e9bd8ef4060a976327db93aca7d [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:46:47.947438 32070 raft_consensus.cc:2454] T a1077e54cb7141f89526fcb0f754ccae P c33d68f5cc3440a3a1b9b0f2b5c6c47e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27920ccf084247bfb6f2c859de429989 in term 1.
I20260810 18:46:47.947455 31863 raft_consensus.cc:2454] T a1077e54cb7141f89526fcb0f754ccae P 13f62e9bd8ef4060a976327db93aca7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27920ccf084247bfb6f2c859de429989 in term 1.
I20260810 18:46:47.947860 31552 leader_election.cc:302] T a1077e54cb7141f89526fcb0f754ccae P 27920ccf084247bfb6f2c859de429989 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27920ccf084247bfb6f2c859de429989, c33d68f5cc3440a3a1b9b0f2b5c6c47e; no voters:
I20260810 18:46:47.948138 32191 raft_consensus.cc:2781] T a1077e54cb7141f89526fcb0f754ccae P 27920ccf084247bfb6f2c859de429989 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:46:47.948623 32191 raft_consensus.cc:686] T a1077e54cb7141f89526fcb0f754ccae P 27920ccf084247bfb6f2c859de429989 [term 1 LEADER]: Becoming Leader. State: Replica: 27920ccf084247bfb6f2c859de429989, State: Running, Role: LEADER
I20260810 18:46:47.949098 32191 consensus_queue.cc:237] T a1077e54cb7141f89526fcb0f754ccae P 27920ccf084247bfb6f2c859de429989 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c33d68f5cc3440a3a1b9b0f2b5c6c47e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36373 } } peers { permanent_uuid: "13f62e9bd8ef4060a976327db93aca7d" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43867 } } peers { permanent_uuid: "27920ccf084247bfb6f2c859de429989" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 33471 } }
I20260810 18:46:47.954258 31431 catalog_manager.cc:5495] T a1077e54cb7141f89526fcb0f754ccae P 27920ccf084247bfb6f2c859de429989 reported cstate change: term changed from 0 to 1, leader changed from <none> to 27920ccf084247bfb6f2c859de429989 (127.4.228.193). New cstate: current_term: 1 leader_uuid: "27920ccf084247bfb6f2c859de429989" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c33d68f5cc3440a3a1b9b0f2b5c6c47e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13f62e9bd8ef4060a976327db93aca7d" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27920ccf084247bfb6f2c859de429989" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 33471 } health_report { overall_health: HEALTHY } } }
I20260810 18:46:47.971179 32175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:47.965779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43408 (local address 127.4.228.193:33471)
0810 18:46:47.965967 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:47.965971 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:47.966140 (+ 169us) server_negotiation.cc:407] Connection header received
0810 18:46:47.966316 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:47.966320 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:47.966406 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:47.966573 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:46:47.966910 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:47.967598 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:47.968983 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:47.969430 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:47.969523 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:47.970053 (+ 530us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:46:47.970730 (+ 677us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:46:47.970881 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:47.970927 (+ 46us) server_negotiation.cc:299] Negotiation successful
0810 18:46:47.970988 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 18:46:47.980070 31863 raft_consensus.cc:1261] T a1077e54cb7141f89526fcb0f754ccae P 13f62e9bd8ef4060a976327db93aca7d [term 1 FOLLOWER]: Refusing update from remote peer 27920ccf084247bfb6f2c859de429989: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:46:47.980069 32070 raft_consensus.cc:1261] T a1077e54cb7141f89526fcb0f754ccae P c33d68f5cc3440a3a1b9b0f2b5c6c47e [term 1 FOLLOWER]: Refusing update from remote peer 27920ccf084247bfb6f2c859de429989: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:46:47.981145 32195 consensus_queue.cc:1035] T a1077e54cb7141f89526fcb0f754ccae P 27920ccf084247bfb6f2c859de429989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13f62e9bd8ef4060a976327db93aca7d" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:46:47.981608 32191 consensus_queue.cc:1035] T a1077e54cb7141f89526fcb0f754ccae P 27920ccf084247bfb6f2c859de429989 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c33d68f5cc3440a3a1b9b0f2b5c6c47e" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:46:47.992530 32199 mvcc.cc:205] Tried to move back new op lower bound from 7317043642274717696 to 7317043642163245056. Current Snapshot: MvccSnapshot[applied={T|T < 7317043642274717696}]
I20260810 18:46:47.995059 32202 mvcc.cc:205] Tried to move back new op lower bound from 7317043642274717696 to 7317043642163245056. Current Snapshot: MvccSnapshot[applied={T|T < 7317043642274717696}]
I20260810 18:46:48.007702 31432 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:46:48.008056 31432 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:46:48.012682 31432 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 509d883077e848e1b284c451d2efa6fc: Sending DeleteTablet for 3 replicas of tablet a1077e54cb7141f89526fcb0f754ccae
I20260810 18:46:48.014281 31630 tablet_service.cc:1514] Processing DeleteTablet for tablet a1077e54cb7141f89526fcb0f754ccae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:46:48 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:43394
I20260810 18:46:48.015192 31830 tablet_service.cc:1514] Processing DeleteTablet for tablet a1077e54cb7141f89526fcb0f754ccae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:46:48 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:47168
I20260810 18:46:48.015375 32214 tablet_replica.cc:330] T a1077e54cb7141f89526fcb0f754ccae P 27920ccf084247bfb6f2c859de429989: stopping tablet replica
I20260810 18:46:48.014331 32038 tablet_service.cc:1514] Processing DeleteTablet for tablet a1077e54cb7141f89526fcb0f754ccae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:46:48 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:48362
I20260810 18:46:48.016117 32214 raft_consensus.cc:2229] T a1077e54cb7141f89526fcb0f754ccae P 27920ccf084247bfb6f2c859de429989 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:46:48.016347 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 31524
I20260810 18:46:48.017001 32214 raft_consensus.cc:2258] T a1077e54cb7141f89526fcb0f754ccae P 27920ccf084247bfb6f2c859de429989 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:46:48.017863 32216 tablet_replica.cc:330] T a1077e54cb7141f89526fcb0f754ccae P c33d68f5cc3440a3a1b9b0f2b5c6c47e: stopping tablet replica
I20260810 18:46:48.018388 32216 raft_consensus.cc:2229] T a1077e54cb7141f89526fcb0f754ccae P c33d68f5cc3440a3a1b9b0f2b5c6c47e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:46:48.019042 32216 raft_consensus.cc:2258] T a1077e54cb7141f89526fcb0f754ccae P c33d68f5cc3440a3a1b9b0f2b5c6c47e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:46:48.022744 32214 ts_tablet_manager.cc:1905] T a1077e54cb7141f89526fcb0f754ccae P 27920ccf084247bfb6f2c859de429989: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:46:48.024773 32216 ts_tablet_manager.cc:1905] T a1077e54cb7141f89526fcb0f754ccae P c33d68f5cc3440a3a1b9b0f2b5c6c47e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:46:48.029635 32215 tablet_replica.cc:330] T a1077e54cb7141f89526fcb0f754ccae P 13f62e9bd8ef4060a976327db93aca7d: stopping tablet replica
I20260810 18:46:48.030318 32215 raft_consensus.cc:2229] T a1077e54cb7141f89526fcb0f754ccae P 13f62e9bd8ef4060a976327db93aca7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:46:48.031134 32214 ts_tablet_manager.cc:1918] T a1077e54cb7141f89526fcb0f754ccae P 27920ccf084247bfb6f2c859de429989: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:46:48.031342 32214 log.cc:1196] T a1077e54cb7141f89526fcb0f754ccae P 27920ccf084247bfb6f2c859de429989: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-0/wal/wals/a1077e54cb7141f89526fcb0f754ccae
I20260810 18:46:48.031371 32215 raft_consensus.cc:2258] T a1077e54cb7141f89526fcb0f754ccae P 13f62e9bd8ef4060a976327db93aca7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:46:48.031935 32214 ts_tablet_manager.cc:1939] T a1077e54cb7141f89526fcb0f754ccae P 27920ccf084247bfb6f2c859de429989: Deleting consensus metadata
I20260810 18:46:48.033003 32216 ts_tablet_manager.cc:1918] T a1077e54cb7141f89526fcb0f754ccae P c33d68f5cc3440a3a1b9b0f2b5c6c47e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:46:48.033208 32216 log.cc:1196] T a1077e54cb7141f89526fcb0f754ccae P c33d68f5cc3440a3a1b9b0f2b5c6c47e: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-2/wal/wals/a1077e54cb7141f89526fcb0f754ccae
I20260810 18:46:48.033797 32216 ts_tablet_manager.cc:1939] T a1077e54cb7141f89526fcb0f754ccae P c33d68f5cc3440a3a1b9b0f2b5c6c47e: Deleting consensus metadata
I20260810 18:46:48.034453 31406 catalog_manager.cc:4841] TS 27920ccf084247bfb6f2c859de429989 (127.4.228.193:33471): tablet a1077e54cb7141f89526fcb0f754ccae (table test-table [id=e82133475eaa4e49aca1dc7eca681218]) successfully deleted
I20260810 18:46:48.035772 31412 catalog_manager.cc:4841] TS c33d68f5cc3440a3a1b9b0f2b5c6c47e (127.4.228.195:36373): tablet a1077e54cb7141f89526fcb0f754ccae (table test-table [id=e82133475eaa4e49aca1dc7eca681218]) successfully deleted
I20260810 18:46:48.037230 32215 ts_tablet_manager.cc:1905] T a1077e54cb7141f89526fcb0f754ccae P 13f62e9bd8ef4060a976327db93aca7d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:46:48.042218 32215 ts_tablet_manager.cc:1918] T a1077e54cb7141f89526fcb0f754ccae P 13f62e9bd8ef4060a976327db93aca7d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:46:48.042375 32215 log.cc:1196] T a1077e54cb7141f89526fcb0f754ccae P 13f62e9bd8ef4060a976327db93aca7d: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387267874959-5011-0/minicluster-data/ts-1/wal/wals/a1077e54cb7141f89526fcb0f754ccae
I20260810 18:46:48.042925 32215 ts_tablet_manager.cc:1939] T a1077e54cb7141f89526fcb0f754ccae P 13f62e9bd8ef4060a976327db93aca7d: Deleting consensus metadata
I20260810 18:46:48.044633 31412 catalog_manager.cc:4841] TS 13f62e9bd8ef4060a976327db93aca7d (127.4.228.194:43867): tablet a1077e54cb7141f89526fcb0f754ccae (table test-table [id=e82133475eaa4e49aca1dc7eca681218]) successfully deleted
Aug 10 18:46:48 dist-test-slave-7kzw krb5kdc[31367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.193@KRBTEST.COM
I20260810 18:46:48.053877 32175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:48.019961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43424 (local address 127.4.228.193:33471)
0810 18:46:48.020168 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:48.020174 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:48.028507 (+ 8333us) server_negotiation.cc:407] Connection header received
0810 18:46:48.028841 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:48.028846 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:48.028947 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:48.029106 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:48.034271 (+ 5165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:48.036156 (+ 1885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:48.038730 (+ 2574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:48.039584 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:48.039722 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:48.046559 (+ 6837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:48.046580 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:48.046598 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:46:48.046655 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:48.050036 (+ 3381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:48.051256 (+ 1220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:48.051265 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:48.051269 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:48.051537 (+ 268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:48.052113 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:48.052117 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:48.052120 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:48.052603 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:48.052843 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:48.053326 (+ 483us) server_negotiation.cc:299] Negotiation successful
0810 18:46:48.053545 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 18:46:48.054889 31720 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:46:48.176051 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 31524
I20260810 18:46:48.209026 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 31735
Aug 10 18:46:48 dist-test-slave-7kzw krb5kdc[31367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.194@KRBTEST.COM
I20260810 18:46:48.227042 32176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:48.209886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47182 (local address 127.4.228.194:43867)
0810 18:46:48.210147 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:48.210154 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:48.210275 (+ 121us) server_negotiation.cc:407] Connection header received
0810 18:46:48.210565 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:48.210570 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:48.210650 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:48.210788 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:48.212355 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:48.213300 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:48.214882 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:48.215325 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:48.215479 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:48.220304 (+ 4825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:48.220320 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:48.220339 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:46:48.220376 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:48.223756 (+ 3380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:48.224617 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:48.224633 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:48.224635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:48.224773 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:48.225223 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:48.225229 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:48.225233 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:48.225674 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:48.226022 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:48.226374 (+ 352us) server_negotiation.cc:299] Negotiation successful
0810 18:46:48.226680 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 18:46:48.227825 31920 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:46:48.322331 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 31735
I20260810 18:46:48.347896 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 31936
Aug 10 18:46:48 dist-test-slave-7kzw krb5kdc[31367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.195@KRBTEST.COM
I20260810 18:46:48.367748 32173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:48.348726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48378 (local address 127.4.228.195:36373)
0810 18:46:48.348913 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:48.348918 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:48.348941 (+ 23us) server_negotiation.cc:407] Connection header received
0810 18:46:48.349128 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:48.349133 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:48.349201 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:48.349290 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:48.350871 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:48.351591 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:48.352919 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:48.353325 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:48.353420 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:48.361617 (+ 8197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:48.361638 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:48.361658 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:46:48.361697 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:48.364717 (+ 3020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:48.365464 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:48.365472 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:48.365475 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:48.365623 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:48.366037 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:48.366042 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:48.366045 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:48.366431 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:48.366797 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:48.367150 (+ 353us) server_negotiation.cc:299] Negotiation successful
0810 18:46:48.367394 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 18:46:48.368414 32141 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:46:48.458532 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 31936
I20260810 18:46:48.485567 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 31386
I20260810 18:46:48.487008 31481 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:46:48.599803 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 31386
2026-08-10T18:46:48Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4923 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-10T18:46:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:46:48Z Disabled control of system clock
I20260810 18:46:48.701337 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:34035
--webserver_interface=127.4.228.254
--webserver_port=0
--builtin_ntp_servers=127.4.228.212:33263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:34035
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 18:46:49.064064 32228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:46:49.064380 32228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:46:49.064550 32228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:46:49.071920 32228 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:46:49.072026 32228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:46:49.072130 32228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:46:49.072242 32228 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:46:49.072327 32228 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:46:49.081569 32228 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:33263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:34035
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:34035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:46:49.082962 32228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:46:49.084940 32228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:46:49.097218 32234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:49.097291 32236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:49.097187 32238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:46:49.102945 32228 server_base.cc:1034] running on GCE node
I20260810 18:46:49.103909 32228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:46:49.105423 32228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:46:49.106665 32228 hybrid_clock.cc:648] HybridClock initialized: now 1786387609106597 us; error 55 us; skew 500 ppm
I20260810 18:46:49.107040 32228 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:46:49.109643 32228 webserver.cc:460] Webserver started at http://127.4.228.254:41537/ using document root <none> and password file <none>
I20260810 18:46:49.110421 32228 fs_manager.cc:359] Metadata directory not provided
I20260810 18:46:49.110600 32228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:46:49.110957 32228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:46:49.114077 32228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "6bd684d585c649bf9df60e6b79706700"
format_stamp: "Formatted at 2026-08-10 18:46:49 on dist-test-slave-7kzw"
server_key: "33cc56141307152e9f995d9f9169616f"
server_key_iv: "759b199e8cb27093f27dc6494d4c6f39"
server_key_version: "encryptionkey@0"
I20260810 18:46:49.114967 32228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "6bd684d585c649bf9df60e6b79706700"
format_stamp: "Formatted at 2026-08-10 18:46:49 on dist-test-slave-7kzw"
server_key: "33cc56141307152e9f995d9f9169616f"
server_key_iv: "759b199e8cb27093f27dc6494d4c6f39"
server_key_version: "encryptionkey@0"
I20260810 18:46:49.120625 32228 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 18:46:49.124310 32250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:49.125692 32228 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:46:49.125907 32228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "6bd684d585c649bf9df60e6b79706700"
format_stamp: "Formatted at 2026-08-10 18:46:49 on dist-test-slave-7kzw"
server_key: "33cc56141307152e9f995d9f9169616f"
server_key_iv: "759b199e8cb27093f27dc6494d4c6f39"
server_key_version: "encryptionkey@0"
I20260810 18:46:49.126170 32228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:46:49.144404 32228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:46:49.145241 32228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:46:49.145556 32228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:46:49.165273 32228 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:34035
I20260810 18:46:49.165303 32327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:34035 every 8 connection(s)
I20260810 18:46:49.166725 32228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:46:49.170794 32328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:46:49.172811 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 32228
I20260810 18:46:49.173014 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:46:49.173504 5011 external_mini_cluster.cc:1428] Setting key 19e67c3e392d3f04b5b377b5bb434b45
I20260810 18:46:49.182561 32328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bd684d585c649bf9df60e6b79706700: Bootstrap starting.
I20260810 18:46:49.185765 32328 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6bd684d585c649bf9df60e6b79706700: Neither blocks nor log segments found. Creating new log.
I20260810 18:46:49.187045 32328 log.cc:824] T 00000000000000000000000000000000 P 6bd684d585c649bf9df60e6b79706700: Log is configured to *not* fsync() on all Append() calls
I20260810 18:46:49.187345 32332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:49.175548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34872 (local address 127.4.228.254:34035)
0810 18:46:49.176641 (+ 1093us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:49.176652 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:49.176700 (+ 48us) server_negotiation.cc:407] Connection header received
0810 18:46:49.178413 (+ 1713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:49.178436 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:49.178950 (+ 514us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:49.179408 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:49.180008 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:49.181296 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:49.182697 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:49.183388 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:49.183592 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:49.184833 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:49.184842 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:49.184855 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:46:49.185391 (+ 536us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:49.185485 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:46:49.185507 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:49.185633 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:49.185868 (+ 235us) server_negotiation.cc:299] Negotiation successful
0810 18:46:49.186247 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":384,"threads_started":1}
I20260810 18:46:49.190169 32328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bd684d585c649bf9df60e6b79706700: No bootstrap required, opened a new log
I20260810 18:46:49.197654 32328 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6bd684d585c649bf9df60e6b79706700 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bd684d585c649bf9df60e6b79706700" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 34035 } }
I20260810 18:46:49.198031 32328 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6bd684d585c649bf9df60e6b79706700 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:46:49.198167 32328 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6bd684d585c649bf9df60e6b79706700 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bd684d585c649bf9df60e6b79706700, State: Initialized, Role: FOLLOWER
I20260810 18:46:49.198815 32328 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6bd684d585c649bf9df60e6b79706700 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bd684d585c649bf9df60e6b79706700" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 34035 } }
I20260810 18:46:49.199055 32328 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6bd684d585c649bf9df60e6b79706700 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:46:49.199218 32328 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bd684d585c649bf9df60e6b79706700 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:46:49.199412 32328 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bd684d585c649bf9df60e6b79706700 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:46:49.201936 32328 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bd684d585c649bf9df60e6b79706700 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bd684d585c649bf9df60e6b79706700" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 34035 } }
I20260810 18:46:49.202546 32328 leader_election.cc:302] T 00000000000000000000000000000000 P 6bd684d585c649bf9df60e6b79706700 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6bd684d585c649bf9df60e6b79706700; no voters:
I20260810 18:46:49.203159 32328 leader_election.cc:288] T 00000000000000000000000000000000 P 6bd684d585c649bf9df60e6b79706700 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:46:49.203315 32334 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bd684d585c649bf9df60e6b79706700 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:46:49.204111 32334 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6bd684d585c649bf9df60e6b79706700 [term 1 LEADER]: Becoming Leader. State: Replica: 6bd684d585c649bf9df60e6b79706700, State: Running, Role: LEADER
I20260810 18:46:49.204692 32334 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6bd684d585c649bf9df60e6b79706700 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bd684d585c649bf9df60e6b79706700" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 34035 } }
I20260810 18:46:49.205365 32328 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6bd684d585c649bf9df60e6b79706700 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:46:49.207803 32340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bd684d585c649bf9df60e6b79706700 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bd684d585c649bf9df60e6b79706700. Latest consensus state: current_term: 1 leader_uuid: "6bd684d585c649bf9df60e6b79706700" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bd684d585c649bf9df60e6b79706700" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 34035 } } }
I20260810 18:46:49.207890 32336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bd684d585c649bf9df60e6b79706700 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6bd684d585c649bf9df60e6b79706700" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bd684d585c649bf9df60e6b79706700" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 34035 } } }
I20260810 18:46:49.208252 32340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bd684d585c649bf9df60e6b79706700 [sys.catalog]: This master's current role is: LEADER
I20260810 18:46:49.208292 32336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bd684d585c649bf9df60e6b79706700 [sys.catalog]: This master's current role is: LEADER
I20260810 18:46:49.209367 32346 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:46:49.215848 32346 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:46:49.225342 32346 catalog_manager.cc:1347] Generated new cluster ID: ce2b81065b184cae9def092bc614d225
I20260810 18:46:49.225525 32346 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:46:49.267853 32346 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:46:49.269439 32346 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:46:49.284296 32346 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6bd684d585c649bf9df60e6b79706700: Generated new TSK 0
I20260810 18:46:49.285269 32346 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:46:49.291009 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.193:0
--local_ip_for_outbound_sockets=127.4.228.193
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:34035
--builtin_ntp_servers=127.4.228.212:33263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 18:46:49.653844 32363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:46:49.654203 32363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:46:49.654421 32363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:46:49.662627 32363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:46:49.662765 32363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:46:49.662994 32363 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.193
I20260810 18:46:49.678748 32363 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:33263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:34035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.193
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:46:49.680663 32363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:46:49.683671 32363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:46:49.702661 32369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:49.702611 32376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:49.702617 32371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:46:49.703801 32363 server_base.cc:1034] running on GCE node
I20260810 18:46:49.704475 32363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:46:49.705608 32363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:46:49.706864 32363 hybrid_clock.cc:648] HybridClock initialized: now 1786387609706787 us; error 67 us; skew 500 ppm
I20260810 18:46:49.707213 32363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:46:49.709611 32363 webserver.cc:460] Webserver started at http://127.4.228.193:34797/ using document root <none> and password file <none>
I20260810 18:46:49.710358 32363 fs_manager.cc:359] Metadata directory not provided
I20260810 18:46:49.710515 32363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:46:49.710925 32363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:46:49.714126 32363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-0/data/instance:
uuid: "4d24f8d5fc3743428bb4d81768faf2b4"
format_stamp: "Formatted at 2026-08-10 18:46:49 on dist-test-slave-7kzw"
server_key: "292375a7b6bf0ded9680c4034d5ff5aa"
server_key_iv: "2d24c70186d05f229753152bca014f37"
server_key_version: "encryptionkey@0"
I20260810 18:46:49.714993 32363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance:
uuid: "4d24f8d5fc3743428bb4d81768faf2b4"
format_stamp: "Formatted at 2026-08-10 18:46:49 on dist-test-slave-7kzw"
server_key: "292375a7b6bf0ded9680c4034d5ff5aa"
server_key_iv: "2d24c70186d05f229753152bca014f37"
server_key_version: "encryptionkey@0"
I20260810 18:46:49.720216 32363 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 18:46:49.723851 32385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:49.725142 32363 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:46:49.725317 32363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-0/wal
uuid: "4d24f8d5fc3743428bb4d81768faf2b4"
format_stamp: "Formatted at 2026-08-10 18:46:49 on dist-test-slave-7kzw"
server_key: "292375a7b6bf0ded9680c4034d5ff5aa"
server_key_iv: "2d24c70186d05f229753152bca014f37"
server_key_version: "encryptionkey@0"
I20260810 18:46:49.725538 32363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:46:49.753500 32363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:46:49.754520 32363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:46:49.754878 32363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:46:49.755995 32363 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:46:49.757817 32363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:46:49.757946 32363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:49.758071 32363 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:46:49.758177 32363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:49.793354 32363 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.193:39597
I20260810 18:46:49.793421 32561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.193:39597 every 8 connection(s)
I20260810 18:46:49.794857 32363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
I20260810 18:46:49.797299 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 32363
I20260810 18:46:49.797505 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance
I20260810 18:46:49.798166 5011 external_mini_cluster.cc:1428] Setting key 03095f8d9c9527c7bcaaee296775df80
I20260810 18:46:49.801148 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.194:0
--local_ip_for_outbound_sockets=127.4.228.194
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:34035
--builtin_ntp_servers=127.4.228.212:33263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 18:46:49.809864 32565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:49.797673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:49501 (local address 127.4.228.254:34035)
0810 18:46:49.798188 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:49.798195 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:49.799790 (+ 1595us) server_negotiation.cc:407] Connection header received
0810 18:46:49.801755 (+ 1965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:49.801772 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:49.801868 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:49.801996 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:49.803221 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:49.804404 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:49.806256 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:49.807043 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:49.807201 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:49.808825 (+ 1624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:49.808834 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:49.808842 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:46:49.808871 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:49.808925 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:46:49.808945 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:49.809049 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:49.809279 (+ 230us) server_negotiation.cc:299] Negotiation successful
0810 18:46:49.809397 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":242,"threads_started":1}
I20260810 18:46:49.810853 32567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:49.798441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:34035 (local address 127.4.228.193:49501)
0810 18:46:49.799626 (+ 1185us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:49.799674 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:49.801466 (+ 1792us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:49.802219 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:49.802227 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:49.802317 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:46:49.803033 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:49.803051 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:49.804595 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:49.804599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:49.806029 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:49.806043 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:49.807297 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:49.807303 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:49.807637 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:49.808432 (+ 795us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:46:49.808434 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:46:49.808482 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:46:49.808566 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:46:49.809054 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:46:49.809063 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:46:49.809069 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 18:46:49.809232 (+ 163us) client_negotiation.cc:240] Negotiation successful
0810 18:46:49.809538 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":358,"threads_started":1}
I20260810 18:46:49.812639 32562 heartbeater.cc:347] Connected to a master server at 127.4.228.254:34035
I20260810 18:46:49.812994 32562 heartbeater.cc:464] Registering TS with master...
I20260810 18:46:49.813977 32562 heartbeater.cc:511] Master 127.4.228.254:34035 requested a full tablet report, sending...
I20260810 18:46:49.816275 32275 ts_manager.cc:194] Registered new tserver with Master: 4d24f8d5fc3743428bb4d81768faf2b4 (127.4.228.193:39597)
I20260810 18:46:49.817762 32275 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.228.193:49501
W20260810 18:46:50.156134 32569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:46:50.156499 32569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:46:50.156720 32569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:46:50.164175 32569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:46:50.164350 32569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:46:50.164520 32569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.194
I20260810 18:46:50.173782 32569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:33263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:34035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.194
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:46:50.175238 32569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:46:50.177116 32569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:46:50.191352 32578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:50.191251 32577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:50.191285 32581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:46:50.196554 32569 server_base.cc:1034] running on GCE node
I20260810 18:46:50.197346 32569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:46:50.198608 32569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:46:50.199898 32569 hybrid_clock.cc:648] HybridClock initialized: now 1786387610199817 us; error 58 us; skew 500 ppm
I20260810 18:46:50.200326 32569 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:46:50.203111 32569 webserver.cc:460] Webserver started at http://127.4.228.194:35531/ using document root <none> and password file <none>
I20260810 18:46:50.203959 32569 fs_manager.cc:359] Metadata directory not provided
I20260810 18:46:50.204129 32569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:46:50.204531 32569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:46:50.208082 32569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-1/data/instance:
uuid: "7b5cc6b3b4d14643a728b9a0e7b8b221"
format_stamp: "Formatted at 2026-08-10 18:46:50 on dist-test-slave-7kzw"
server_key: "02554ca024530288a8326c74395174f1"
server_key_iv: "fd5220005d9438f2936cb486fee9ae7a"
server_key_version: "encryptionkey@0"
I20260810 18:46:50.209111 32569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance:
uuid: "7b5cc6b3b4d14643a728b9a0e7b8b221"
format_stamp: "Formatted at 2026-08-10 18:46:50 on dist-test-slave-7kzw"
server_key: "02554ca024530288a8326c74395174f1"
server_key_iv: "fd5220005d9438f2936cb486fee9ae7a"
server_key_version: "encryptionkey@0"
I20260810 18:46:50.215435 32569 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260810 18:46:50.219303 32592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:50.220854 32569 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:46:50.221082 32569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-1/wal
uuid: "7b5cc6b3b4d14643a728b9a0e7b8b221"
format_stamp: "Formatted at 2026-08-10 18:46:50 on dist-test-slave-7kzw"
server_key: "02554ca024530288a8326c74395174f1"
server_key_iv: "fd5220005d9438f2936cb486fee9ae7a"
server_key_version: "encryptionkey@0"
I20260810 18:46:50.221342 32569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:46:50.239948 32569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:46:50.240971 32569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:46:50.241353 32569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:46:50.242622 32569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:46:50.244354 32569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:46:50.244508 32569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:50.244663 32569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:46:50.244766 32569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:50.283339 32569 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.194:43475
I20260810 18:46:50.283403 32767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.194:43475 every 8 connection(s)
I20260810 18:46:50.284852 32569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
I20260810 18:46:50.287101 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 32569
I20260810 18:46:50.287345 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance
I20260810 18:46:50.287875 5011 external_mini_cluster.cc:1428] Setting key 287f668a0e7928a28218465e137b5edb
I20260810 18:46:50.291759 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.195:0
--local_ip_for_outbound_sockets=127.4.228.195
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:34035
--builtin_ntp_servers=127.4.228.212:33263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 18:46:50.302286 32565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:50.288250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:50011 (local address 127.4.228.254:34035)
0810 18:46:50.288509 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:50.288517 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:50.290310 (+ 1793us) server_negotiation.cc:407] Connection header received
0810 18:46:50.293380 (+ 3070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:50.293384 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:50.293456 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:50.293551 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:50.295224 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:50.296015 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:50.298919 (+ 2904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:50.299500 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:50.299616 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:50.301453 (+ 1837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:50.301459 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:50.301462 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:46:50.301484 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:50.301520 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:46:50.301533 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:50.301606 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:50.301914 (+ 308us) server_negotiation.cc:299] Negotiation successful
0810 18:46:50.302003 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260810 18:46:50.303957 303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:50.288878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:34035 (local address 127.4.228.194:50011)
0810 18:46:50.290120 (+ 1242us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:50.290186 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:50.293025 (+ 2839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:50.293866 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:50.293880 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:50.294055 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:46:50.295003 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:50.295046 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:50.296195 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:50.296204 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:50.298695 (+ 2491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:50.298716 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:50.299763 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:50.299774 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:50.300163 (+ 389us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:50.301020 (+ 857us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:46:50.301026 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:46:50.301098 (+ 72us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:46:50.301216 (+ 118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:46:50.301641 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:46:50.301654 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:46:50.301664 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 18:46:50.301872 (+ 208us) client_negotiation.cc:240] Negotiation successful
0810 18:46:50.302358 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"thread_start_us":462,"threads_started":1}
I20260810 18:46:50.305935 301 heartbeater.cc:347] Connected to a master server at 127.4.228.254:34035
I20260810 18:46:50.306396 301 heartbeater.cc:464] Registering TS with master...
I20260810 18:46:50.307822 301 heartbeater.cc:511] Master 127.4.228.254:34035 requested a full tablet report, sending...
I20260810 18:46:50.309841 32275 ts_manager.cc:194] Registered new tserver with Master: 7b5cc6b3b4d14643a728b9a0e7b8b221 (127.4.228.194:43475)
I20260810 18:46:50.311043 32275 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.228.194:50011
W20260810 18:46:50.649524 304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:46:50.649808 304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:46:50.649959 304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:46:50.657351 304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:46:50.657459 304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:46:50.657629 304 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.195
I20260810 18:46:50.667189 304 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:33263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:34035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.195
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:46:50.668700 304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:46:50.670625 304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:46:50.683995 312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:50.683985 313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:50.683920 315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:46:50.684717 304 server_base.cc:1034] running on GCE node
I20260810 18:46:50.685294 304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:46:50.686210 304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:46:50.687453 304 hybrid_clock.cc:648] HybridClock initialized: now 1786387610687393 us; error 59 us; skew 500 ppm
I20260810 18:46:50.687819 304 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:46:50.690316 304 webserver.cc:460] Webserver started at http://127.4.228.195:35183/ using document root <none> and password file <none>
I20260810 18:46:50.691121 304 fs_manager.cc:359] Metadata directory not provided
I20260810 18:46:50.691259 304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:46:50.691612 304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:46:50.694528 304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-2/data/instance:
uuid: "4ad15e1bba0449e0a14771261f76bed8"
format_stamp: "Formatted at 2026-08-10 18:46:50 on dist-test-slave-7kzw"
server_key: "93fd75c455a8b4b93d4024a9dba5ba80"
server_key_iv: "4ba85e4b5865f113d1db7227409d3cc6"
server_key_version: "encryptionkey@0"
I20260810 18:46:50.695384 304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance:
uuid: "4ad15e1bba0449e0a14771261f76bed8"
format_stamp: "Formatted at 2026-08-10 18:46:50 on dist-test-slave-7kzw"
server_key: "93fd75c455a8b4b93d4024a9dba5ba80"
server_key_iv: "4ba85e4b5865f113d1db7227409d3cc6"
server_key_version: "encryptionkey@0"
I20260810 18:46:50.700691 304 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260810 18:46:50.704396 323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:50.705876 304 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:46:50.706077 304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-2/wal
uuid: "4ad15e1bba0449e0a14771261f76bed8"
format_stamp: "Formatted at 2026-08-10 18:46:50 on dist-test-slave-7kzw"
server_key: "93fd75c455a8b4b93d4024a9dba5ba80"
server_key_iv: "4ba85e4b5865f113d1db7227409d3cc6"
server_key_version: "encryptionkey@0"
I20260810 18:46:50.706310 304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:46:50.718869 304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:46:50.719735 304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:46:50.720049 304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:46:50.721021 304 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:46:50.722548 304 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:46:50.722680 304 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:50.722817 304 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:46:50.722914 304 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:50.758313 304 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.195:45113
I20260810 18:46:50.758412 481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.195:45113 every 8 connection(s)
I20260810 18:46:50.759747 304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
I20260810 18:46:50.760426 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 304
I20260810 18:46:50.760694 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance
I20260810 18:46:50.761349 5011 external_mini_cluster.cc:1428] Setting key b9d75fee7f829e93176a0e83f18f90aa
I20260810 18:46:50.773315 32565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:50.763060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:50599 (local address 127.4.228.254:34035)
0810 18:46:50.763231 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:50.763236 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:50.765006 (+ 1770us) server_negotiation.cc:407] Connection header received
0810 18:46:50.767002 (+ 1996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:50.767006 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:50.767073 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:50.767166 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:50.768281 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:50.769030 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:50.770640 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:50.771049 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:50.771150 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:50.772600 (+ 1450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:50.772604 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:50.772607 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:46:50.772637 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:50.772674 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:46:50.772686 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:50.772759 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:50.772968 (+ 209us) server_negotiation.cc:299] Negotiation successful
0810 18:46:50.773048 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 18:46:50.774302 485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:50.763625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:34035 (local address 127.4.228.195:50599)
0810 18:46:50.764859 (+ 1234us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:50.764908 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:50.766719 (+ 1811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:50.767389 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:50.767396 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:50.767492 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:46:50.768112 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:50.768131 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:50.769191 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:50.769195 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:50.770434 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:50.770447 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:50.771289 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:50.771296 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:50.771541 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:50.772287 (+ 746us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:46:50.772291 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:46:50.772336 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:46:50.772412 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:46:50.772775 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:46:50.772783 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:46:50.772789 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 18:46:50.772936 (+ 147us) client_negotiation.cc:240] Negotiation successful
0810 18:46:50.773223 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"thread_start_us":427,"threads_started":1}
I20260810 18:46:50.775914 483 heartbeater.cc:347] Connected to a master server at 127.4.228.254:34035
I20260810 18:46:50.776264 483 heartbeater.cc:464] Registering TS with master...
I20260810 18:46:50.777031 483 heartbeater.cc:511] Master 127.4.228.254:34035 requested a full tablet report, sending...
I20260810 18:46:50.778540 32275 ts_manager.cc:194] Registered new tserver with Master: 4ad15e1bba0449e0a14771261f76bed8 (127.4.228.195:45113)
I20260810 18:46:50.779368 32275 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.228.195:50599
I20260810 18:46:50.789465 5011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:46:50.800302 32565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:50.794003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34884 (local address 127.4.228.254:34035)
0810 18:46:50.794178 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:50.794183 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:50.794797 (+ 614us) server_negotiation.cc:407] Connection header received
0810 18:46:50.795010 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:50.795022 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:50.795085 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:50.795216 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:50.795765 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:50.796469 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:50.797931 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:50.798355 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:50.798508 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:50.799566 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:50.799570 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:50.799574 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:46:50.799591 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:50.799626 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:46:50.799641 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:50.799743 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:50.799896 (+ 153us) server_negotiation.cc:299] Negotiation successful
0810 18:46:50.799991 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 18:46:50.804375 32275 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34884:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:46:50.811969 32275 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:46:50.821792 32562 heartbeater.cc:503] Master 127.4.228.254:34035 was elected leader, sending a full tablet report...
I20260810 18:46:50.840480 503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:50.825133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:45113 (local address 127.0.0.1:44874)
0810 18:46:50.826000 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:50.826070 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:50.826253 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:50.827601 (+ 1348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:50.827609 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:50.827646 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:46:50.828083 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:50.828100 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:50.832322 (+ 4222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:50.832332 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:50.835864 (+ 3532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:50.835891 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:50.839276 (+ 3385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:50.839286 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:50.839563 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:50.839627 (+ 64us) client_negotiation.cc:769] Sending connection context
0810 18:46:50.839790 (+ 163us) client_negotiation.cc:240] Negotiation successful
0810 18:46:50.840003 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":357,"threads_started":1}
I20260810 18:46:50.841425 505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:50.825461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44874 (local address 127.4.228.195:45113)
0810 18:46:50.826407 (+ 946us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:50.826417 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:50.826460 (+ 43us) server_negotiation.cc:407] Connection header received
0810 18:46:50.826700 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:50.826711 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:50.826987 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:50.827228 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:46:50.828348 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:50.832073 (+ 3725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:50.836256 (+ 4183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:50.838440 (+ 2184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:50.838654 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:50.839093 (+ 439us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:50.840782 (+ 1689us) server_negotiation.cc:299] Negotiation successful
0810 18:46:50.840995 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":756,"thread_start_us":605,"threads_started":1}
I20260810 18:46:50.846366 394 tablet_service.cc:1467] Processing CreateTablet for tablet 0395a11417d7477aa12fdc32721c4794 (DEFAULT_TABLE table=test-table [id=56362d640e3c44eaae740ff305739368]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:46:50.848387 394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0395a11417d7477aa12fdc32721c4794. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:46:50.850943 507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:50.826802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:39597 (local address 127.0.0.1:52428)
0810 18:46:50.840558 (+ 13756us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:50.840864 (+ 306us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:50.841009 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:50.842496 (+ 1487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:50.842502 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:50.842545 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:46:50.842904 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:50.842919 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:50.845130 (+ 2211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:50.845136 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:50.847649 (+ 2513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:50.847668 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:50.849999 (+ 2331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:50.850007 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:50.850194 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:50.850228 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 18:46:50.850317 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 18:46:50.850487 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13585,"mutex_wait_us":1,"thread_start_us":13401,"threads_started":1}
I20260810 18:46:50.851778 508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:50.827150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52428 (local address 127.4.228.193:39597)
0810 18:46:50.839831 (+ 12681us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:50.839841 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:50.841262 (+ 1421us) server_negotiation.cc:407] Connection header received
0810 18:46:50.841383 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:50.841394 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:50.841943 (+ 549us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:50.842187 (+ 244us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:46:50.843108 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:50.844939 (+ 1831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:50.847896 (+ 2957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:50.849401 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:50.849545 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:50.849851 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:50.851199 (+ 1348us) server_negotiation.cc:299] Negotiation successful
0810 18:46:50.851390 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12448,"thread_start_us":12702,"threads_started":1}
I20260810 18:46:50.851557 506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:50.826389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:43475 (local address 127.0.0.1:42282)
0810 18:46:50.827001 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:50.827020 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:50.841812 (+ 14792us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:50.843111 (+ 1299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:50.843116 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:50.843147 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:46:50.843511 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:50.843526 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:50.845907 (+ 2381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:50.845913 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:50.848343 (+ 2430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:50.848360 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:50.850557 (+ 2197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:50.850565 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:50.850841 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:50.850876 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 18:46:50.850964 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 18:46:50.851162 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":1013,"threads_started":1}
I20260810 18:46:50.852701 509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:50.827379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42282 (local address 127.4.228.194:43475)
0810 18:46:50.842402 (+ 15023us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:50.842413 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:50.842445 (+ 32us) server_negotiation.cc:407] Connection header received
0810 18:46:50.842583 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:50.842594 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:50.842851 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:50.843756 (+ 905us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:46:50.844009 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:50.845717 (+ 1708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:50.848626 (+ 2909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:50.850279 (+ 1653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:50.851724 (+ 1445us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:50.851991 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:50.852089 (+ 98us) server_negotiation.cc:299] Negotiation successful
0810 18:46:50.852286 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14818,"thread_start_us":17007,"threads_started":1}
I20260810 18:46:50.856928 32460 tablet_service.cc:1467] Processing CreateTablet for tablet 0395a11417d7477aa12fdc32721c4794 (DEFAULT_TABLE table=test-table [id=56362d640e3c44eaae740ff305739368]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:46:50.856927 32662 tablet_service.cc:1467] Processing CreateTablet for tablet 0395a11417d7477aa12fdc32721c4794 (DEFAULT_TABLE table=test-table [id=56362d640e3c44eaae740ff305739368]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:46:50.858626 32460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0395a11417d7477aa12fdc32721c4794. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:46:50.859064 32662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0395a11417d7477aa12fdc32721c4794. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:46:50.862344 512 tablet_bootstrap.cc:492] T 0395a11417d7477aa12fdc32721c4794 P 4ad15e1bba0449e0a14771261f76bed8: Bootstrap starting.
I20260810 18:46:50.866009 512 tablet_bootstrap.cc:654] T 0395a11417d7477aa12fdc32721c4794 P 4ad15e1bba0449e0a14771261f76bed8: Neither blocks nor log segments found. Creating new log.
I20260810 18:46:50.867172 512 log.cc:824] T 0395a11417d7477aa12fdc32721c4794 P 4ad15e1bba0449e0a14771261f76bed8: Log is configured to *not* fsync() on all Append() calls
I20260810 18:46:50.870007 512 tablet_bootstrap.cc:492] T 0395a11417d7477aa12fdc32721c4794 P 4ad15e1bba0449e0a14771261f76bed8: No bootstrap required, opened a new log
I20260810 18:46:50.870313 512 ts_tablet_manager.cc:1397] T 0395a11417d7477aa12fdc32721c4794 P 4ad15e1bba0449e0a14771261f76bed8: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260810 18:46:50.875550 517 tablet_bootstrap.cc:492] T 0395a11417d7477aa12fdc32721c4794 P 4d24f8d5fc3743428bb4d81768faf2b4: Bootstrap starting.
I20260810 18:46:50.876407 515 tablet_bootstrap.cc:492] T 0395a11417d7477aa12fdc32721c4794 P 7b5cc6b3b4d14643a728b9a0e7b8b221: Bootstrap starting.
I20260810 18:46:50.879868 512 raft_consensus.cc:348] T 0395a11417d7477aa12fdc32721c4794 P 4ad15e1bba0449e0a14771261f76bed8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad15e1bba0449e0a14771261f76bed8" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45113 } } peers { permanent_uuid: "7b5cc6b3b4d14643a728b9a0e7b8b221" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43475 } } peers { permanent_uuid: "4d24f8d5fc3743428bb4d81768faf2b4" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 39597 } }
I20260810 18:46:50.880364 512 raft_consensus.cc:374] T 0395a11417d7477aa12fdc32721c4794 P 4ad15e1bba0449e0a14771261f76bed8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:46:50.880478 512 raft_consensus.cc:729] T 0395a11417d7477aa12fdc32721c4794 P 4ad15e1bba0449e0a14771261f76bed8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ad15e1bba0449e0a14771261f76bed8, State: Initialized, Role: FOLLOWER
I20260810 18:46:50.880751 517 tablet_bootstrap.cc:654] T 0395a11417d7477aa12fdc32721c4794 P 4d24f8d5fc3743428bb4d81768faf2b4: Neither blocks nor log segments found. Creating new log.
I20260810 18:46:50.881453 515 tablet_bootstrap.cc:654] T 0395a11417d7477aa12fdc32721c4794 P 7b5cc6b3b4d14643a728b9a0e7b8b221: Neither blocks nor log segments found. Creating new log.
I20260810 18:46:50.881383 512 consensus_queue.cc:260] T 0395a11417d7477aa12fdc32721c4794 P 4ad15e1bba0449e0a14771261f76bed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad15e1bba0449e0a14771261f76bed8" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45113 } } peers { permanent_uuid: "7b5cc6b3b4d14643a728b9a0e7b8b221" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43475 } } peers { permanent_uuid: "4d24f8d5fc3743428bb4d81768faf2b4" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 39597 } }
I20260810 18:46:50.882239 517 log.cc:824] T 0395a11417d7477aa12fdc32721c4794 P 4d24f8d5fc3743428bb4d81768faf2b4: Log is configured to *not* fsync() on all Append() calls
I20260810 18:46:50.882850 483 heartbeater.cc:503] Master 127.4.228.254:34035 was elected leader, sending a full tablet report...
I20260810 18:46:50.883065 515 log.cc:824] T 0395a11417d7477aa12fdc32721c4794 P 7b5cc6b3b4d14643a728b9a0e7b8b221: Log is configured to *not* fsync() on all Append() calls
I20260810 18:46:50.883442 512 ts_tablet_manager.cc:1428] T 0395a11417d7477aa12fdc32721c4794 P 4ad15e1bba0449e0a14771261f76bed8: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
I20260810 18:46:50.886811 517 tablet_bootstrap.cc:492] T 0395a11417d7477aa12fdc32721c4794 P 4d24f8d5fc3743428bb4d81768faf2b4: No bootstrap required, opened a new log
I20260810 18:46:50.886826 515 tablet_bootstrap.cc:492] T 0395a11417d7477aa12fdc32721c4794 P 7b5cc6b3b4d14643a728b9a0e7b8b221: No bootstrap required, opened a new log
I20260810 18:46:50.887135 517 ts_tablet_manager.cc:1397] T 0395a11417d7477aa12fdc32721c4794 P 4d24f8d5fc3743428bb4d81768faf2b4: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260810 18:46:50.887135 515 ts_tablet_manager.cc:1397] T 0395a11417d7477aa12fdc32721c4794 P 7b5cc6b3b4d14643a728b9a0e7b8b221: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.002s
I20260810 18:46:50.899471 517 raft_consensus.cc:348] T 0395a11417d7477aa12fdc32721c4794 P 4d24f8d5fc3743428bb4d81768faf2b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad15e1bba0449e0a14771261f76bed8" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45113 } } peers { permanent_uuid: "7b5cc6b3b4d14643a728b9a0e7b8b221" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43475 } } peers { permanent_uuid: "4d24f8d5fc3743428bb4d81768faf2b4" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 39597 } }
I20260810 18:46:50.899467 515 raft_consensus.cc:348] T 0395a11417d7477aa12fdc32721c4794 P 7b5cc6b3b4d14643a728b9a0e7b8b221 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad15e1bba0449e0a14771261f76bed8" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45113 } } peers { permanent_uuid: "7b5cc6b3b4d14643a728b9a0e7b8b221" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43475 } } peers { permanent_uuid: "4d24f8d5fc3743428bb4d81768faf2b4" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 39597 } }
I20260810 18:46:50.900023 517 raft_consensus.cc:374] T 0395a11417d7477aa12fdc32721c4794 P 4d24f8d5fc3743428bb4d81768faf2b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:46:50.900027 515 raft_consensus.cc:374] T 0395a11417d7477aa12fdc32721c4794 P 7b5cc6b3b4d14643a728b9a0e7b8b221 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:46:50.900153 517 raft_consensus.cc:729] T 0395a11417d7477aa12fdc32721c4794 P 4d24f8d5fc3743428bb4d81768faf2b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d24f8d5fc3743428bb4d81768faf2b4, State: Initialized, Role: FOLLOWER
I20260810 18:46:50.900153 515 raft_consensus.cc:729] T 0395a11417d7477aa12fdc32721c4794 P 7b5cc6b3b4d14643a728b9a0e7b8b221 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b5cc6b3b4d14643a728b9a0e7b8b221, State: Initialized, Role: FOLLOWER
I20260810 18:46:50.900959 517 consensus_queue.cc:260] T 0395a11417d7477aa12fdc32721c4794 P 4d24f8d5fc3743428bb4d81768faf2b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad15e1bba0449e0a14771261f76bed8" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45113 } } peers { permanent_uuid: "7b5cc6b3b4d14643a728b9a0e7b8b221" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43475 } } peers { permanent_uuid: "4d24f8d5fc3743428bb4d81768faf2b4" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 39597 } }
I20260810 18:46:50.900928 515 consensus_queue.cc:260] T 0395a11417d7477aa12fdc32721c4794 P 7b5cc6b3b4d14643a728b9a0e7b8b221 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad15e1bba0449e0a14771261f76bed8" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45113 } } peers { permanent_uuid: "7b5cc6b3b4d14643a728b9a0e7b8b221" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43475 } } peers { permanent_uuid: "4d24f8d5fc3743428bb4d81768faf2b4" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 39597 } }
I20260810 18:46:50.902714 301 heartbeater.cc:503] Master 127.4.228.254:34035 was elected leader, sending a full tablet report...
I20260810 18:46:50.903268 517 ts_tablet_manager.cc:1428] T 0395a11417d7477aa12fdc32721c4794 P 4d24f8d5fc3743428bb4d81768faf2b4: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
I20260810 18:46:50.903403 515 ts_tablet_manager.cc:1428] T 0395a11417d7477aa12fdc32721c4794 P 7b5cc6b3b4d14643a728b9a0e7b8b221: Time spent starting tablet: real 0.016s user 0.014s sys 0.002s
I20260810 18:46:50.957862 518 raft_consensus.cc:482] T 0395a11417d7477aa12fdc32721c4794 P 4ad15e1bba0449e0a14771261f76bed8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:46:50.958227 518 raft_consensus.cc:504] T 0395a11417d7477aa12fdc32721c4794 P 4ad15e1bba0449e0a14771261f76bed8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad15e1bba0449e0a14771261f76bed8" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45113 } } peers { permanent_uuid: "7b5cc6b3b4d14643a728b9a0e7b8b221" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43475 } } peers { permanent_uuid: "4d24f8d5fc3743428bb4d81768faf2b4" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 39597 } }
I20260810 18:46:50.960461 518 leader_election.cc:288] T 0395a11417d7477aa12fdc32721c4794 P 4ad15e1bba0449e0a14771261f76bed8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b5cc6b3b4d14643a728b9a0e7b8b221 (127.4.228.194:43475), 4d24f8d5fc3743428bb4d81768faf2b4 (127.4.228.193:39597)
I20260810 18:46:50.967490 485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:50.960733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:43475 (local address 127.4.228.195:53429)
0810 18:46:50.961086 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:50.961110 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:50.961295 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:50.961830 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:50.961836 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:50.961869 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:46:50.962114 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:50.962129 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:50.963378 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:50.963381 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:50.965303 (+ 1922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:50.965318 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:50.966816 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:50.966822 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:50.966949 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:50.966975 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 18:46:50.967061 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 18:46:50.967171 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211}
I20260810 18:46:50.967566 509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:50.960990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:53429 (local address 127.4.228.194:43475)
0810 18:46:50.961203 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:50.961210 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:50.961233 (+ 23us) server_negotiation.cc:407] Connection header received
0810 18:46:50.961522 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:50.961528 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:50.961637 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:50.961751 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:46:50.962327 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:50.963209 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:50.965499 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:50.966611 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:50.966780 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:50.966860 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:50.967123 (+ 263us) server_negotiation.cc:299] Negotiation successful
0810 18:46:50.967288 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 18:46:50.968321 524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:50.960954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:39597 (local address 127.4.228.195:44541)
0810 18:46:50.961893 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:50.961914 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:50.962052 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:50.962515 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:50.962523 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:50.962562 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:46:50.962878 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:50.962886 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:50.964033 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:50.964039 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:50.965898 (+ 1859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:50.965910 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:50.967500 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:50.967512 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:50.967751 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:50.967784 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 18:46:50.967903 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 18:46:50.968029 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":798,"thread_start_us":529,"threads_started":1}
I20260810 18:46:50.968340 508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:50.961113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:44541 (local address 127.4.228.193:39597)
0810 18:46:50.961346 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:50.961355 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:50.962021 (+ 666us) server_negotiation.cc:407] Connection header received
0810 18:46:50.962181 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:50.962185 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:50.962296 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:50.962400 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:46:50.963056 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:50.963854 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:50.966083 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:50.967288 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:50.967457 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:50.967522 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:50.967966 (+ 444us) server_negotiation.cc:299] Negotiation successful
0810 18:46:50.968093 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 18:46:50.968582 32696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0395a11417d7477aa12fdc32721c4794" candidate_uuid: "4ad15e1bba0449e0a14771261f76bed8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b5cc6b3b4d14643a728b9a0e7b8b221" is_pre_election: true
I20260810 18:46:50.969249 32696 raft_consensus.cc:2454] T 0395a11417d7477aa12fdc32721c4794 P 7b5cc6b3b4d14643a728b9a0e7b8b221 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ad15e1bba0449e0a14771261f76bed8 in term 0.
I20260810 18:46:50.969484 32492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0395a11417d7477aa12fdc32721c4794" candidate_uuid: "4ad15e1bba0449e0a14771261f76bed8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d24f8d5fc3743428bb4d81768faf2b4" is_pre_election: true
I20260810 18:46:50.970069 32492 raft_consensus.cc:2454] T 0395a11417d7477aa12fdc32721c4794 P 4d24f8d5fc3743428bb4d81768faf2b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ad15e1bba0449e0a14771261f76bed8 in term 0.
I20260810 18:46:50.970031 326 leader_election.cc:302] T 0395a11417d7477aa12fdc32721c4794 P 4ad15e1bba0449e0a14771261f76bed8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ad15e1bba0449e0a14771261f76bed8, 7b5cc6b3b4d14643a728b9a0e7b8b221; no voters:
I20260810 18:46:50.970547 518 raft_consensus.cc:2781] T 0395a11417d7477aa12fdc32721c4794 P 4ad15e1bba0449e0a14771261f76bed8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:46:50.970741 518 raft_consensus.cc:482] T 0395a11417d7477aa12fdc32721c4794 P 4ad15e1bba0449e0a14771261f76bed8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:46:50.970887 518 raft_consensus.cc:3037] T 0395a11417d7477aa12fdc32721c4794 P 4ad15e1bba0449e0a14771261f76bed8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:46:50.973129 518 raft_consensus.cc:504] T 0395a11417d7477aa12fdc32721c4794 P 4ad15e1bba0449e0a14771261f76bed8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad15e1bba0449e0a14771261f76bed8" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45113 } } peers { permanent_uuid: "7b5cc6b3b4d14643a728b9a0e7b8b221" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43475 } } peers { permanent_uuid: "4d24f8d5fc3743428bb4d81768faf2b4" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 39597 } }
I20260810 18:46:50.973795 518 leader_election.cc:288] T 0395a11417d7477aa12fdc32721c4794 P 4ad15e1bba0449e0a14771261f76bed8 [CANDIDATE]: Term 1 election: Requested vote from peers 7b5cc6b3b4d14643a728b9a0e7b8b221 (127.4.228.194:43475), 4d24f8d5fc3743428bb4d81768faf2b4 (127.4.228.193:39597)
I20260810 18:46:50.974234 32696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0395a11417d7477aa12fdc32721c4794" candidate_uuid: "4ad15e1bba0449e0a14771261f76bed8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b5cc6b3b4d14643a728b9a0e7b8b221"
I20260810 18:46:50.974257 32492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0395a11417d7477aa12fdc32721c4794" candidate_uuid: "4ad15e1bba0449e0a14771261f76bed8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d24f8d5fc3743428bb4d81768faf2b4"
I20260810 18:46:50.974454 32492 raft_consensus.cc:3037] T 0395a11417d7477aa12fdc32721c4794 P 4d24f8d5fc3743428bb4d81768faf2b4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:46:50.974442 32696 raft_consensus.cc:3037] T 0395a11417d7477aa12fdc32721c4794 P 7b5cc6b3b4d14643a728b9a0e7b8b221 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:46:50.976711 32492 raft_consensus.cc:2454] T 0395a11417d7477aa12fdc32721c4794 P 4d24f8d5fc3743428bb4d81768faf2b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ad15e1bba0449e0a14771261f76bed8 in term 1.
I20260810 18:46:50.976722 32696 raft_consensus.cc:2454] T 0395a11417d7477aa12fdc32721c4794 P 7b5cc6b3b4d14643a728b9a0e7b8b221 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ad15e1bba0449e0a14771261f76bed8 in term 1.
I20260810 18:46:50.977188 327 leader_election.cc:302] T 0395a11417d7477aa12fdc32721c4794 P 4ad15e1bba0449e0a14771261f76bed8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ad15e1bba0449e0a14771261f76bed8, 4d24f8d5fc3743428bb4d81768faf2b4; no voters:
I20260810 18:46:50.977552 518 raft_consensus.cc:2781] T 0395a11417d7477aa12fdc32721c4794 P 4ad15e1bba0449e0a14771261f76bed8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:46:50.978004 518 raft_consensus.cc:686] T 0395a11417d7477aa12fdc32721c4794 P 4ad15e1bba0449e0a14771261f76bed8 [term 1 LEADER]: Becoming Leader. State: Replica: 4ad15e1bba0449e0a14771261f76bed8, State: Running, Role: LEADER
I20260810 18:46:50.978583 518 consensus_queue.cc:237] T 0395a11417d7477aa12fdc32721c4794 P 4ad15e1bba0449e0a14771261f76bed8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad15e1bba0449e0a14771261f76bed8" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45113 } } peers { permanent_uuid: "7b5cc6b3b4d14643a728b9a0e7b8b221" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43475 } } peers { permanent_uuid: "4d24f8d5fc3743428bb4d81768faf2b4" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 39597 } }
I20260810 18:46:50.983605 32273 catalog_manager.cc:5495] T 0395a11417d7477aa12fdc32721c4794 P 4ad15e1bba0449e0a14771261f76bed8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ad15e1bba0449e0a14771261f76bed8 (127.4.228.195). New cstate: current_term: 1 leader_uuid: "4ad15e1bba0449e0a14771261f76bed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad15e1bba0449e0a14771261f76bed8" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 45113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7b5cc6b3b4d14643a728b9a0e7b8b221" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d24f8d5fc3743428bb4d81768faf2b4" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 39597 } health_report { overall_health: UNKNOWN } } }
I20260810 18:46:51.008698 505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:51.002935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44890 (local address 127.4.228.195:45113)
0810 18:46:51.003139 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:51.003144 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:51.003163 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:46:51.003260 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:51.003264 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:51.003353 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:51.003482 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:46:51.003860 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:51.004675 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:51.006253 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:51.006859 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:51.006979 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:51.007569 (+ 590us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:46:51.008154 (+ 585us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:46:51.008284 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:51.008402 (+ 118us) server_negotiation.cc:299] Negotiation successful
0810 18:46:51.008474 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260810 18:46:51.011806 484 tablet.cc:2369] T 0395a11417d7477aa12fdc32721c4794 P 4ad15e1bba0449e0a14771261f76bed8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:46:51.018062 32492 raft_consensus.cc:1261] T 0395a11417d7477aa12fdc32721c4794 P 4d24f8d5fc3743428bb4d81768faf2b4 [term 1 FOLLOWER]: Refusing update from remote peer 4ad15e1bba0449e0a14771261f76bed8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:46:51.018069 32696 raft_consensus.cc:1261] T 0395a11417d7477aa12fdc32721c4794 P 7b5cc6b3b4d14643a728b9a0e7b8b221 [term 1 FOLLOWER]: Refusing update from remote peer 4ad15e1bba0449e0a14771261f76bed8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:46:51.019184 525 consensus_queue.cc:1035] T 0395a11417d7477aa12fdc32721c4794 P 4ad15e1bba0449e0a14771261f76bed8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d24f8d5fc3743428bb4d81768faf2b4" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 39597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:46:51.019538 518 consensus_queue.cc:1035] T 0395a11417d7477aa12fdc32721c4794 P 4ad15e1bba0449e0a14771261f76bed8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b5cc6b3b4d14643a728b9a0e7b8b221" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 43475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:46:51.048780 32274 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:46:51.049140 32274 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:46:51.053861 32274 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6bd684d585c649bf9df60e6b79706700: Sending DeleteTablet for 3 replicas of tablet 0395a11417d7477aa12fdc32721c4794
I20260810 18:46:51.055270 394 tablet_service.cc:1514] Processing DeleteTablet for tablet 0395a11417d7477aa12fdc32721c4794 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:46:51 UTC) from {username='slave'} at 127.0.0.1:44874
I20260810 18:46:51.055361 32662 tablet_service.cc:1514] Processing DeleteTablet for tablet 0395a11417d7477aa12fdc32721c4794 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:46:51 UTC) from {username='slave'} at 127.0.0.1:42282
I20260810 18:46:51.055423 32460 tablet_service.cc:1514] Processing DeleteTablet for tablet 0395a11417d7477aa12fdc32721c4794 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:46:51 UTC) from {username='slave'} at 127.0.0.1:52428
I20260810 18:46:51.059302 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 32363
I20260810 18:46:51.068630 550 tablet_replica.cc:330] T 0395a11417d7477aa12fdc32721c4794 P 4d24f8d5fc3743428bb4d81768faf2b4: stopping tablet replica
I20260810 18:46:51.069177 550 raft_consensus.cc:2229] T 0395a11417d7477aa12fdc32721c4794 P 4d24f8d5fc3743428bb4d81768faf2b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:46:51.069797 550 raft_consensus.cc:2258] T 0395a11417d7477aa12fdc32721c4794 P 4d24f8d5fc3743428bb4d81768faf2b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:46:51.070604 547 tablet_replica.cc:330] T 0395a11417d7477aa12fdc32721c4794 P 4ad15e1bba0449e0a14771261f76bed8: stopping tablet replica
I20260810 18:46:51.071141 547 raft_consensus.cc:2229] T 0395a11417d7477aa12fdc32721c4794 P 4ad15e1bba0449e0a14771261f76bed8 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:46:51.071954 547 raft_consensus.cc:2258] T 0395a11417d7477aa12fdc32721c4794 P 4ad15e1bba0449e0a14771261f76bed8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:46:51.072039 550 ts_tablet_manager.cc:1905] T 0395a11417d7477aa12fdc32721c4794 P 4d24f8d5fc3743428bb4d81768faf2b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:46:51.074481 547 ts_tablet_manager.cc:1905] T 0395a11417d7477aa12fdc32721c4794 P 4ad15e1bba0449e0a14771261f76bed8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:46:51.080013 550 ts_tablet_manager.cc:1918] T 0395a11417d7477aa12fdc32721c4794 P 4d24f8d5fc3743428bb4d81768faf2b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:46:51.080204 550 log.cc:1196] T 0395a11417d7477aa12fdc32721c4794 P 4d24f8d5fc3743428bb4d81768faf2b4: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-0/wal/wals/0395a11417d7477aa12fdc32721c4794
I20260810 18:46:51.080842 550 ts_tablet_manager.cc:1939] T 0395a11417d7477aa12fdc32721c4794 P 4d24f8d5fc3743428bb4d81768faf2b4: Deleting consensus metadata
I20260810 18:46:51.083072 32257 catalog_manager.cc:4841] TS 4d24f8d5fc3743428bb4d81768faf2b4 (127.4.228.193:39597): tablet 0395a11417d7477aa12fdc32721c4794 (table test-table [id=56362d640e3c44eaae740ff305739368]) successfully deleted
I20260810 18:46:51.084053 547 ts_tablet_manager.cc:1918] T 0395a11417d7477aa12fdc32721c4794 P 4ad15e1bba0449e0a14771261f76bed8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:46:51.084321 547 log.cc:1196] T 0395a11417d7477aa12fdc32721c4794 P 4ad15e1bba0449e0a14771261f76bed8: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-2/wal/wals/0395a11417d7477aa12fdc32721c4794
I20260810 18:46:51.085062 547 ts_tablet_manager.cc:1939] T 0395a11417d7477aa12fdc32721c4794 P 4ad15e1bba0449e0a14771261f76bed8: Deleting consensus metadata
I20260810 18:46:51.085480 548 tablet_replica.cc:330] T 0395a11417d7477aa12fdc32721c4794 P 7b5cc6b3b4d14643a728b9a0e7b8b221: stopping tablet replica
I20260810 18:46:51.086052 548 raft_consensus.cc:2229] T 0395a11417d7477aa12fdc32721c4794 P 7b5cc6b3b4d14643a728b9a0e7b8b221 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:46:51.086745 548 raft_consensus.cc:2258] T 0395a11417d7477aa12fdc32721c4794 P 7b5cc6b3b4d14643a728b9a0e7b8b221 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:46:51.087100 32256 catalog_manager.cc:4841] TS 4ad15e1bba0449e0a14771261f76bed8 (127.4.228.195:45113): tablet 0395a11417d7477aa12fdc32721c4794 (table test-table [id=56362d640e3c44eaae740ff305739368]) successfully deleted
I20260810 18:46:51.089329 548 ts_tablet_manager.cc:1905] T 0395a11417d7477aa12fdc32721c4794 P 7b5cc6b3b4d14643a728b9a0e7b8b221: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:46:51.091677 508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:51.061223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52436 (local address 127.4.228.193:39597)
0810 18:46:51.061471 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:51.061480 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:51.082256 (+ 20776us) server_negotiation.cc:407] Connection header received
0810 18:46:51.082871 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:51.082878 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:51.082999 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:51.083084 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:51.083961 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:51.085099 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:51.087309 (+ 2210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:51.087968 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:51.088116 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:51.089812 (+ 1696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:51.089825 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:51.089843 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:46:51.090722 (+ 879us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:51.090793 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:46:51.090821 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:51.090957 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:51.091139 (+ 182us) server_negotiation.cc:299] Negotiation successful
0810 18:46:51.091268 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 18:46:51.092603 32555 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:46:51.098006 548 ts_tablet_manager.cc:1918] T 0395a11417d7477aa12fdc32721c4794 P 7b5cc6b3b4d14643a728b9a0e7b8b221: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:46:51.098203 548 log.cc:1196] T 0395a11417d7477aa12fdc32721c4794 P 7b5cc6b3b4d14643a728b9a0e7b8b221: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387267874959-5011-0/minicluster-data/ts-1/wal/wals/0395a11417d7477aa12fdc32721c4794
I20260810 18:46:51.098899 548 ts_tablet_manager.cc:1939] T 0395a11417d7477aa12fdc32721c4794 P 7b5cc6b3b4d14643a728b9a0e7b8b221: Deleting consensus metadata
I20260810 18:46:51.101087 32256 catalog_manager.cc:4841] TS 7b5cc6b3b4d14643a728b9a0e7b8b221 (127.4.228.194:43475): tablet 0395a11417d7477aa12fdc32721c4794 (table test-table [id=56362d640e3c44eaae740ff305739368]) successfully deleted
I20260810 18:46:51.195855 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 32363
I20260810 18:46:51.230345 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 32569
I20260810 18:46:51.238879 509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:51.231305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42290 (local address 127.4.228.194:43475)
0810 18:46:51.231529 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:51.231538 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:51.231651 (+ 113us) server_negotiation.cc:407] Connection header received
0810 18:46:51.231854 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:51.231861 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:51.231996 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:51.232129 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:51.232512 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:51.233673 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:51.235060 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:51.235748 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:51.235899 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:51.237048 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:51.237061 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:51.237084 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:46:51.237927 (+ 843us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:51.238003 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:46:51.238033 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:51.238164 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:51.238295 (+ 131us) server_negotiation.cc:299] Negotiation successful
0810 18:46:51.238422 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260810 18:46:51.239727 32762 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:46:51.328467 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 32569
I20260810 18:46:51.355126 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 304
I20260810 18:46:51.364156 505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:51.356097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44904 (local address 127.4.228.195:45113)
0810 18:46:51.356366 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:51.356373 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:51.356419 (+ 46us) server_negotiation.cc:407] Connection header received
0810 18:46:51.356496 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:51.356502 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:51.356618 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:51.356736 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:51.357139 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:51.358316 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:51.359875 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:51.360588 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:51.360724 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:51.361897 (+ 1173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:51.361912 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:51.361933 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:46:51.362877 (+ 944us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:51.362947 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:46:51.362974 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:51.363093 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:51.363644 (+ 551us) server_negotiation.cc:299] Negotiation successful
0810 18:46:51.363756 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260810 18:46:51.365150 477 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:46:51.486572 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 304
I20260810 18:46:51.516383 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 32228
I20260810 18:46:51.517771 32320 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:46:51.629433 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 32228
2026-08-10T18:46:51Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3028 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:46:51 dist-test-slave-7kzw krb5kdc[560](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:46:51 dist-test-slave-7kzw krb5kdc[560](info): set up 2 sockets
Aug 10 18:46:51 dist-test-slave-7kzw krb5kdc[560](info): commencing operation
krb5kdc: starting...
W20260810 18:46:52.980293 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.297s user 0.002s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:46:53 dist-test-slave-7kzw krb5kdc[560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:46:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:46:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.228.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
I20260810 18:46:53.310319 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:33263
--webserver_interface=127.4.228.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:36821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:33263
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:46:53.688270 578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:46:53.688642 578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:46:53.688805 578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:46:53.696208 578 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:46:53.696365 578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:46:53.696475 578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:46:53.696579 578 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:46:53.696656 578 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:46:53.706815 578 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:36821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:33263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:33263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:46:53.708274 578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:46:53.709735 578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:46:53.722573 587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:53.722616 591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:53.722702 588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:46:53.723734 578 server_base.cc:1034] running on GCE node
I20260810 18:46:53.724586 578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:46:53.726104 578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:46:53.727339 578 hybrid_clock.cc:648] HybridClock initialized: now 1786387613727289 us; error 36 us; skew 500 ppm
I20260810 18:46:53.727725 578 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:46:53 dist-test-slave-7kzw krb5kdc[560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387613, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:46:53.730978 578 init.cc:377] Logged in from keytab as kudu/127.4.228.254@KRBTEST.COM (short username kudu)
I20260810 18:46:53.732661 578 webserver.cc:460] Webserver started at http://127.4.228.254:40629/ using document root <none> and password file <none>
I20260810 18:46:53.733407 578 fs_manager.cc:359] Metadata directory not provided
I20260810 18:46:53.733551 578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:46:53.733887 578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:46:53.737084 578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "44da3965af404d018912552edfcf787a"
format_stamp: "Formatted at 2026-08-10 18:46:53 on dist-test-slave-7kzw"
server_key: "80a1a66702bcd318e5f2f3acdabd24e1"
server_key_iv: "9a68d8eb2322b5ba1dae19b3a03c518b"
server_key_version: "encryptionkey@0"
I20260810 18:46:53.737828 578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "44da3965af404d018912552edfcf787a"
format_stamp: "Formatted at 2026-08-10 18:46:53 on dist-test-slave-7kzw"
server_key: "80a1a66702bcd318e5f2f3acdabd24e1"
server_key_iv: "9a68d8eb2322b5ba1dae19b3a03c518b"
server_key_version: "encryptionkey@0"
I20260810 18:46:53.743247 578 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260810 18:46:53.747042 603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:53.748435 578 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:46:53.748605 578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "44da3965af404d018912552edfcf787a"
format_stamp: "Formatted at 2026-08-10 18:46:53 on dist-test-slave-7kzw"
server_key: "80a1a66702bcd318e5f2f3acdabd24e1"
server_key_iv: "9a68d8eb2322b5ba1dae19b3a03c518b"
server_key_version: "encryptionkey@0"
I20260810 18:46:53.748793 578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:46:53.781793 578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:46:53.786422 578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:46:53.786821 578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:46:53.807013 578 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:33263
I20260810 18:46:53.807067 691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:33263 every 8 connection(s)
I20260810 18:46:53.808502 578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:46:53.812688 692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:46:53.815351 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 578
I20260810 18:46:53.815578 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:46:53.816062 5011 external_mini_cluster.cc:1428] Setting key aa8b8c4d2896f932cfd8d986f0970ecb
I20260810 18:46:53.824419 692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44da3965af404d018912552edfcf787a: Bootstrap starting.
I20260810 18:46:53.827653 692 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44da3965af404d018912552edfcf787a: Neither blocks nor log segments found. Creating new log.
I20260810 18:46:53.829046 692 log.cc:824] T 00000000000000000000000000000000 P 44da3965af404d018912552edfcf787a: Log is configured to *not* fsync() on all Append() calls
Aug 10 18:46:53 dist-test-slave-7kzw krb5kdc[560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:46:53.832410 692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44da3965af404d018912552edfcf787a: No bootstrap required, opened a new log
I20260810 18:46:53.840920 695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:53.818090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47950 (local address 127.4.228.254:33263)
0810 18:46:53.818954 (+ 864us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:53.818966 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:53.819009 (+ 43us) server_negotiation.cc:407] Connection header received
0810 18:46:53.820706 (+ 1697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:53.820730 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:53.821148 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:53.821543 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:53.823273 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:53.824623 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:53.825998 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:53.826759 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:53.826958 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:53.831900 (+ 4942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:53.831913 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:53.831933 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:46:53.831979 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:53.836157 (+ 4178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:53.836900 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:53.836909 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:53.836917 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:53.837153 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:53.837515 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:53.837520 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:53.837524 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:53.838266 (+ 742us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:53.838620 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:53.839036 (+ 416us) server_negotiation.cc:299] Negotiation successful
0810 18:46:53.839513 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":414,"threads_started":1}
I20260810 18:46:53.843168 692 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44da3965af404d018912552edfcf787a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44da3965af404d018912552edfcf787a" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 33263 } }
I20260810 18:46:53.843557 692 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44da3965af404d018912552edfcf787a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:46:53.843709 692 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44da3965af404d018912552edfcf787a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44da3965af404d018912552edfcf787a, State: Initialized, Role: FOLLOWER
I20260810 18:46:53.844336 692 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44da3965af404d018912552edfcf787a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44da3965af404d018912552edfcf787a" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 33263 } }
I20260810 18:46:53.844556 692 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44da3965af404d018912552edfcf787a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:46:53.844667 692 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44da3965af404d018912552edfcf787a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:46:53.844789 692 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44da3965af404d018912552edfcf787a [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:46:53.847103 692 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44da3965af404d018912552edfcf787a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44da3965af404d018912552edfcf787a" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 33263 } }
I20260810 18:46:53.847565 692 leader_election.cc:302] T 00000000000000000000000000000000 P 44da3965af404d018912552edfcf787a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44da3965af404d018912552edfcf787a; no voters:
I20260810 18:46:53.848155 692 leader_election.cc:288] T 00000000000000000000000000000000 P 44da3965af404d018912552edfcf787a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:46:53.848317 697 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44da3965af404d018912552edfcf787a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:46:53.849928 692 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44da3965af404d018912552edfcf787a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:46:53.852515 697 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44da3965af404d018912552edfcf787a [term 1 LEADER]: Becoming Leader. State: Replica: 44da3965af404d018912552edfcf787a, State: Running, Role: LEADER
I20260810 18:46:53.853175 697 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44da3965af404d018912552edfcf787a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44da3965af404d018912552edfcf787a" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 33263 } }
I20260810 18:46:53.856518 698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44da3965af404d018912552edfcf787a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44da3965af404d018912552edfcf787a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44da3965af404d018912552edfcf787a" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 33263 } } }
I20260810 18:46:53.857126 698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44da3965af404d018912552edfcf787a [sys.catalog]: This master's current role is: LEADER
I20260810 18:46:53.856531 700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44da3965af404d018912552edfcf787a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44da3965af404d018912552edfcf787a. Latest consensus state: current_term: 1 leader_uuid: "44da3965af404d018912552edfcf787a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44da3965af404d018912552edfcf787a" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 33263 } } }
I20260810 18:46:53.857920 700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44da3965af404d018912552edfcf787a [sys.catalog]: This master's current role is: LEADER
W20260810 18:46:53.862145 718 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 44da3965af404d018912552edfcf787a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:46:53.862257 718 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:46:53.862403 719 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:46:53.864681 719 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:46:53.874085 719 catalog_manager.cc:1347] Generated new cluster ID: 147f71f1e86e471d9a04ee6e3d630d81
I20260810 18:46:53.874195 719 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:46:53.889951 719 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:46:53.891909 719 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:46:53.905685 719 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44da3965af404d018912552edfcf787a: Generated new TSK 0
I20260810 18:46:53.906736 719 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.228.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab.
Entry for principal kudu/127.4.228.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab.
I20260810 18:46:54.002341 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.193:0
--local_ip_for_outbound_sockets=127.4.228.193
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:33263
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:36821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:46:54.385569 729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:46:54.385865 729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:46:54.386031 729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:46:54.393383 729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:46:54.393498 729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:46:54.393680 729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.193
I20260810 18:46:54.403600 729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:36821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:33263
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.193
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:46:54.405107 729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:46:54.406904 729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:46:54.420809 734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:54.420857 738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:54.420735 735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:46:54.421664 729 server_base.cc:1034] running on GCE node
I20260810 18:46:54.422333 729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:46:54.423374 729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:46:54.424579 729 hybrid_clock.cc:648] HybridClock initialized: now 1786387614424496 us; error 79 us; skew 500 ppm
I20260810 18:46:54.424939 729 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:46:54 dist-test-slave-7kzw krb5kdc[560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387614, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:46:54.428328 729 init.cc:377] Logged in from keytab as kudu/127.4.228.193@KRBTEST.COM (short username kudu)
I20260810 18:46:54.429760 729 webserver.cc:460] Webserver started at http://127.4.228.193:36533/ using document root <none> and password file <none>
I20260810 18:46:54.430405 729 fs_manager.cc:359] Metadata directory not provided
I20260810 18:46:54.430567 729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:46:54.430872 729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:46:54.433710 729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-0/data/instance:
uuid: "40564b1a8574472baf18aad60dcd95bc"
format_stamp: "Formatted at 2026-08-10 18:46:54 on dist-test-slave-7kzw"
server_key: "d64bb01bb8537fd6fe4ef05845d284f6"
server_key_iv: "53d8a2a3a2927aeb05601cc71de18dc6"
server_key_version: "encryptionkey@0"
I20260810 18:46:54.434509 729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance:
uuid: "40564b1a8574472baf18aad60dcd95bc"
format_stamp: "Formatted at 2026-08-10 18:46:54 on dist-test-slave-7kzw"
server_key: "d64bb01bb8537fd6fe4ef05845d284f6"
server_key_iv: "53d8a2a3a2927aeb05601cc71de18dc6"
server_key_version: "encryptionkey@0"
I20260810 18:46:54.439849 729 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 18:46:54.443367 746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:54.444716 729 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:46:54.444902 729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-0/wal
uuid: "40564b1a8574472baf18aad60dcd95bc"
format_stamp: "Formatted at 2026-08-10 18:46:54 on dist-test-slave-7kzw"
server_key: "d64bb01bb8537fd6fe4ef05845d284f6"
server_key_iv: "53d8a2a3a2927aeb05601cc71de18dc6"
server_key_version: "encryptionkey@0"
I20260810 18:46:54.445108 729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:46:54.475416 729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:46:54.479795 729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:46:54.480101 729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:46:54.481163 729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:46:54.482848 729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:46:54.482968 729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:54.483103 729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:46:54.483186 729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:54.526365 729 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.193:46489
I20260810 18:46:54.526417 909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.193:46489 every 8 connection(s)
I20260810 18:46:54.527791 729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
I20260810 18:46:54.533278 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 729
I20260810 18:46:54.533486 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance
I20260810 18:46:54.533982 5011 external_mini_cluster.cc:1428] Setting key fc619a31927955fcd464da726ff8aedc
Aug 10 18:46:54 dist-test-slave-7kzw krb5kdc[560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387614, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:46:54.548280 913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:54.530613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:36929 (local address 127.4.228.254:33263)
0810 18:46:54.531094 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:54.531098 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:54.531990 (+ 892us) server_negotiation.cc:407] Connection header received
0810 18:46:54.533978 (+ 1988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:54.533985 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:54.534083 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:54.534207 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:54.535798 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:54.536493 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:54.538055 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:54.538450 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:54.538648 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:54.542343 (+ 3695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:54.542348 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:54.542351 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:46:54.542383 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:54.545108 (+ 2725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:54.545845 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:54.545853 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:54.545856 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:54.545985 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:54.546429 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:54.546435 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:54.546438 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:54.546771 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:54.546930 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:54.547639 (+ 709us) server_negotiation.cc:299] Negotiation successful
0810 18:46:54.547868 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":252,"threads_started":1}
I20260810 18:46:54.549360 914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:54.531100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:33263 (local address 127.4.228.193:36929)
0810 18:46:54.531823 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:54.531870 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:54.533692 (+ 1822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:54.534454 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:54.534507 (+ 53us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:54.535052 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:46:54.535617 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:54.535636 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:54.536656 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:54.536662 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:54.537895 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:54.537908 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:54.538743 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:54.538750 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:54.538985 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:54.539696 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:46:54.539719 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:46:54.542113 (+ 2394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:46:54.545269 (+ 3156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:46:54.545279 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:46:54.545294 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:46:54.545721 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:46:54.546104 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:46:54.546109 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:46:54.546113 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:46:54.546308 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:46:54.546904 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:46:54.546912 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:46:54.547246 (+ 334us) client_negotiation.cc:769] Sending connection context
0810 18:46:54.547542 (+ 296us) client_negotiation.cc:240] Negotiation successful
0810 18:46:54.548085 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":285,"threads_started":1}
I20260810 18:46:54.551134 910 heartbeater.cc:347] Connected to a master server at 127.4.228.254:33263
I20260810 18:46:54.551537 910 heartbeater.cc:464] Registering TS with master...
I20260810 18:46:54.552881 910 heartbeater.cc:511] Master 127.4.228.254:33263 requested a full tablet report, sending...
I20260810 18:46:54.556445 632 ts_manager.cc:194] Registered new tserver with Master: 40564b1a8574472baf18aad60dcd95bc (127.4.228.193:46489)
I20260810 18:46:54.558116 632 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.193@KRBTEST.COM'} at 127.4.228.193:36929
WARNING: no policy specified for kudu/127.4.228.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab.
Entry for principal kudu/127.4.228.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab.
I20260810 18:46:54.608341 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.194:0
--local_ip_for_outbound_sockets=127.4.228.194
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:33263
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:36821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:46:54.961109 917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:46:54.961401 917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:46:54.961496 917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:46:54.968559 917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:46:54.968668 917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:46:54.968779 917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.194
I20260810 18:46:54.978256 917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:36821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:33263
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.194
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:46:54.979671 917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:46:54.980986 917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:46:54.995195 924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:54.995102 929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:54.995162 926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:46:54.996384 917 server_base.cc:1034] running on GCE node
I20260810 18:46:54.997001 917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:46:54.997974 917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:46:54.999235 917 hybrid_clock.cc:648] HybridClock initialized: now 1786387614999190 us; error 57 us; skew 500 ppm
I20260810 18:46:54.999588 917 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:46:54 dist-test-slave-7kzw krb5kdc[560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387614, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:46:55.002943 917 init.cc:377] Logged in from keytab as kudu/127.4.228.194@KRBTEST.COM (short username kudu)
I20260810 18:46:55.004519 917 webserver.cc:460] Webserver started at http://127.4.228.194:43873/ using document root <none> and password file <none>
I20260810 18:46:55.005259 917 fs_manager.cc:359] Metadata directory not provided
I20260810 18:46:55.005409 917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:46:55.005757 917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:46:55.008842 917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-1/data/instance:
uuid: "8cfc6df8c120471f8a48acb1d34263bc"
format_stamp: "Formatted at 2026-08-10 18:46:55 on dist-test-slave-7kzw"
server_key: "d3246ce9a99ab121221f5e0b987977ee"
server_key_iv: "90361f3ba940e20acb3968799f40157c"
server_key_version: "encryptionkey@0"
I20260810 18:46:55.009655 917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance:
uuid: "8cfc6df8c120471f8a48acb1d34263bc"
format_stamp: "Formatted at 2026-08-10 18:46:55 on dist-test-slave-7kzw"
server_key: "d3246ce9a99ab121221f5e0b987977ee"
server_key_iv: "90361f3ba940e20acb3968799f40157c"
server_key_version: "encryptionkey@0"
I20260810 18:46:55.014981 917 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260810 18:46:55.018533 942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:55.019907 917 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:46:55.020082 917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-1/wal
uuid: "8cfc6df8c120471f8a48acb1d34263bc"
format_stamp: "Formatted at 2026-08-10 18:46:55 on dist-test-slave-7kzw"
server_key: "d3246ce9a99ab121221f5e0b987977ee"
server_key_iv: "90361f3ba940e20acb3968799f40157c"
server_key_version: "encryptionkey@0"
I20260810 18:46:55.020292 917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:46:55.033561 917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:46:55.037704 917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:46:55.038013 917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:46:55.039100 917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:46:55.040477 917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:46:55.040623 917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:55.040777 917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:46:55.040856 917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:55.075067 917 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.194:39491
I20260810 18:46:55.075086 1102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.194:39491 every 8 connection(s)
I20260810 18:46:55.076526 917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
I20260810 18:46:55.081382 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 917
I20260810 18:46:55.081691 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance
I20260810 18:46:55.082190 5011 external_mini_cluster.cc:1428] Setting key f90e46c383b09b0b08357421b2535dc4
Aug 10 18:46:55 dist-test-slave-7kzw krb5kdc[560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387614, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:46:55.098165 1106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:55.079365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:53525 (local address 127.4.228.254:33263)
0810 18:46:55.079828 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:55.079833 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:55.080879 (+ 1046us) server_negotiation.cc:407] Connection header received
0810 18:46:55.083034 (+ 2155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:55.083040 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:55.083172 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:55.083302 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:55.085290 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:55.086049 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:55.087732 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:55.088160 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:55.088296 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:55.092144 (+ 3848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:55.092151 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:55.092155 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:46:55.092190 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:55.095028 (+ 2838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:55.095790 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:55.095796 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:55.095798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:55.095921 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:55.096392 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:55.096396 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:55.096398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:55.096673 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:55.096833 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:55.097536 (+ 703us) server_negotiation.cc:299] Negotiation successful
0810 18:46:55.097798 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":227,"threads_started":1}
I20260810 18:46:55.099232 1107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:55.079969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:33263 (local address 127.4.228.194:53525)
0810 18:46:55.080715 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:55.080753 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:55.082688 (+ 1935us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:55.083598 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:55.083608 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:55.084408 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:46:55.085085 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:55.085103 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:55.086226 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:55.086231 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:55.087552 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:55.087566 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:55.088418 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:55.088426 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:55.088658 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:55.089402 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:46:55.089424 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:46:55.091905 (+ 2481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:46:55.095194 (+ 3289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:46:55.095203 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:46:55.095226 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:46:55.095648 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:46:55.096044 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:46:55.096049 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:46:55.096053 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:46:55.096260 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:46:55.096812 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:46:55.096820 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:46:55.097167 (+ 347us) client_negotiation.cc:769] Sending connection context
0810 18:46:55.097436 (+ 269us) client_negotiation.cc:240] Negotiation successful
0810 18:46:55.097984 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":290,"threads_started":1}
I20260810 18:46:55.100798 1103 heartbeater.cc:347] Connected to a master server at 127.4.228.254:33263
I20260810 18:46:55.101141 1103 heartbeater.cc:464] Registering TS with master...
I20260810 18:46:55.101900 1103 heartbeater.cc:511] Master 127.4.228.254:33263 requested a full tablet report, sending...
I20260810 18:46:55.103463 632 ts_manager.cc:194] Registered new tserver with Master: 8cfc6df8c120471f8a48acb1d34263bc (127.4.228.194:39491)
I20260810 18:46:55.104317 632 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.194@KRBTEST.COM'} at 127.4.228.194:53525
WARNING: no policy specified for kudu/127.4.228.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab.
Entry for principal kudu/127.4.228.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab.
I20260810 18:46:55.151989 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.195:0
--local_ip_for_outbound_sockets=127.4.228.195
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:33263
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:36821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:46:55.502524 1110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:46:55.502882 1110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:46:55.503073 1110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:46:55.510277 1110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:46:55.510448 1110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:46:55.510670 1110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.195
I20260810 18:46:55.520761 1110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:36821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:33263
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.195
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:46:55.522235 1110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:46:55.523773 1110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:46:55.537846 1122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:55.537910 1120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:55.537930 1119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:46:55.538750 1110 server_base.cc:1034] running on GCE node
I20260810 18:46:55.539448 1110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:46:55.540431 1110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:46:55.541682 1110 hybrid_clock.cc:648] HybridClock initialized: now 1786387615541619 us; error 90 us; skew 500 ppm
I20260810 18:46:55.542057 1110 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:46:55 dist-test-slave-7kzw krb5kdc[560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387615, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:46:55.545619 1110 init.cc:377] Logged in from keytab as kudu/127.4.228.195@KRBTEST.COM (short username kudu)
I20260810 18:46:55.547209 1110 webserver.cc:460] Webserver started at http://127.4.228.195:36817/ using document root <none> and password file <none>
I20260810 18:46:55.547894 1110 fs_manager.cc:359] Metadata directory not provided
I20260810 18:46:55.548000 1110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:46:55.548261 1110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:46:55.551484 1110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-2/data/instance:
uuid: "d8fed23c259e43ccb6b3e46fed3c4b01"
format_stamp: "Formatted at 2026-08-10 18:46:55 on dist-test-slave-7kzw"
server_key: "ced1f0f6b5e7f928183240d6c041162e"
server_key_iv: "091483d8745519ca3bab9591b881ab3c"
server_key_version: "encryptionkey@0"
I20260810 18:46:55.552248 1110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance:
uuid: "d8fed23c259e43ccb6b3e46fed3c4b01"
format_stamp: "Formatted at 2026-08-10 18:46:55 on dist-test-slave-7kzw"
server_key: "ced1f0f6b5e7f928183240d6c041162e"
server_key_iv: "091483d8745519ca3bab9591b881ab3c"
server_key_version: "encryptionkey@0"
I20260810 18:46:55.558635 1110 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260810 18:46:55.561367 910 heartbeater.cc:503] Master 127.4.228.254:33263 was elected leader, sending a full tablet report...
I20260810 18:46:55.562568 1131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:55.564113 1110 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:46:55.564296 1110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-2/wal
uuid: "d8fed23c259e43ccb6b3e46fed3c4b01"
format_stamp: "Formatted at 2026-08-10 18:46:55 on dist-test-slave-7kzw"
server_key: "ced1f0f6b5e7f928183240d6c041162e"
server_key_iv: "091483d8745519ca3bab9591b881ab3c"
server_key_version: "encryptionkey@0"
I20260810 18:46:55.564512 1110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:46:55.586419 1110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:46:55.591192 1110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:46:55.591509 1110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:46:55.592515 1110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:46:55.593912 1110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:46:55.594026 1110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:55.594182 1110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:46:55.594257 1110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:55.630221 1110 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.195:36229
I20260810 18:46:55.630273 1319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.195:36229 every 8 connection(s)
I20260810 18:46:55.631616 1110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
I20260810 18:46:55.632683 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 1110
I20260810 18:46:55.632881 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance
I20260810 18:46:55.633466 5011 external_mini_cluster.cc:1428] Setting key e4fbdadc9fcdd30232186afcea6b3c04
Aug 10 18:46:55 dist-test-slave-7kzw krb5kdc[560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387615, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:46:55.654441 1324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:55.634698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:51189 (local address 127.4.228.254:33263)
0810 18:46:55.635235 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:55.635241 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:55.636192 (+ 951us) server_negotiation.cc:407] Connection header received
0810 18:46:55.638250 (+ 2058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:55.638255 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:55.638360 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:55.638509 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:55.640058 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:55.640874 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:55.642508 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:55.642949 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:55.643066 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:55.646785 (+ 3719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:55.646790 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:55.646794 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:46:55.646833 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:55.650641 (+ 3808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:55.651450 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:55.651455 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:55.651457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:55.651593 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:55.652043 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:55.652049 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:55.652056 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:55.652422 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:55.652643 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:55.653603 (+ 960us) server_negotiation.cc:299] Negotiation successful
0810 18:46:55.653896 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":260,"threads_started":1}
I20260810 18:46:55.655287 1326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:55.635091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:33263 (local address 127.4.228.195:51189)
0810 18:46:55.636024 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:55.636086 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:55.637987 (+ 1901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:55.638752 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:55.638760 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:55.639302 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:46:55.639883 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:55.639900 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:55.641034 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:55.641041 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:55.642281 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:55.642294 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:55.643205 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:55.643211 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:55.643412 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:55.644168 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:46:55.644196 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:46:55.646573 (+ 2377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:46:55.650793 (+ 4220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:46:55.650801 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:46:55.650813 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:46:55.651326 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:46:55.651700 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:46:55.651704 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:46:55.651706 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:46:55.651887 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:46:55.652577 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:46:55.652588 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:46:55.653118 (+ 530us) client_negotiation.cc:769] Sending connection context
0810 18:46:55.653478 (+ 360us) client_negotiation.cc:240] Negotiation successful
0810 18:46:55.653995 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":368,"threads_started":1}
I20260810 18:46:55.656764 1320 heartbeater.cc:347] Connected to a master server at 127.4.228.254:33263
I20260810 18:46:55.657078 1320 heartbeater.cc:464] Registering TS with master...
I20260810 18:46:55.657809 1320 heartbeater.cc:511] Master 127.4.228.254:33263 requested a full tablet report, sending...
I20260810 18:46:55.659400 632 ts_manager.cc:194] Registered new tserver with Master: d8fed23c259e43ccb6b3e46fed3c4b01 (127.4.228.195:36229)
I20260810 18:46:55.660239 632 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.195@KRBTEST.COM'} at 127.4.228.195:51189
I20260810 18:46:55.662406 5011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:46:55.682758 1324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:55.668958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47952 (local address 127.4.228.254:33263)
0810 18:46:55.669217 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:55.669224 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:55.669618 (+ 394us) server_negotiation.cc:407] Connection header received
0810 18:46:55.669895 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:55.669900 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:55.669980 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:55.670066 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:55.671793 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:55.672546 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:55.674020 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:55.674448 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:55.674624 (+ 176us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:55.677362 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:55.677367 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:55.677370 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:46:55.677403 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:55.679994 (+ 2591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:55.680756 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:55.680760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:55.680762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:55.680897 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:55.681358 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:55.681361 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:55.681364 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:55.681732 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:55.681931 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:55.682279 (+ 348us) server_negotiation.cc:299] Negotiation successful
0810 18:46:55.682422 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 18:46:55.687145 632 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47952:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:46:55.694164 632 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:46:55.735666 1348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:55.710630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40322 (local address 127.4.228.193:46489)
0810 18:46:55.725657 (+ 15027us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:55.725667 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:55.726382 (+ 715us) server_negotiation.cc:407] Connection header received
0810 18:46:55.726518 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:55.726526 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:55.726740 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:55.726904 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:46:55.728181 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:55.729489 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:55.732901 (+ 3412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:55.733993 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:55.734180 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:55.734441 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:55.735173 (+ 732us) server_negotiation.cc:299] Negotiation successful
0810 18:46:55.735341 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14819,"thread_start_us":14630,"threads_started":1}
I20260810 18:46:55.735734 1344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:55.709894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:46489 (local address 127.0.0.1:40322)
0810 18:46:55.726125 (+ 16231us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:55.726144 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:55.726263 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:55.727177 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:55.727185 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:55.727242 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:46:55.727947 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:55.727964 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:55.730028 (+ 2064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:55.730036 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:55.732718 (+ 2682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:55.732732 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:55.734595 (+ 1863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:55.734602 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:55.734819 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:55.734878 (+ 59us) client_negotiation.cc:769] Sending connection context
0810 18:46:55.735025 (+ 147us) client_negotiation.cc:240] Negotiation successful
0810 18:46:55.735241 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16101,"mutex_wait_us":167,"spinlock_wait_cycles":3840,"thread_start_us":15670,"threads_started":1}
I20260810 18:46:55.736367 1346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:55.708501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45636 (local address 127.4.228.194:39491)
0810 18:46:55.724108 (+ 15607us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:55.724123 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:55.726164 (+ 2041us) server_negotiation.cc:407] Connection header received
0810 18:46:55.726301 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:55.726314 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:55.726631 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:55.726859 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:46:55.728188 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:55.729769 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:55.733081 (+ 3312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:55.734606 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:55.734760 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:55.734949 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:55.735846 (+ 897us) server_negotiation.cc:299] Negotiation successful
0810 18:46:55.736028 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15359,"thread_start_us":15197,"threads_started":1}
I20260810 18:46:55.736701 1347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:55.708135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:39491 (local address 127.0.0.1:45636)
0810 18:46:55.725847 (+ 17712us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:55.725873 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:55.726021 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:55.727177 (+ 1156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:55.727185 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:55.727225 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:46:55.727969 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:55.727979 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:55.730000 (+ 2021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:55.730004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:55.732877 (+ 2873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:55.732895 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:55.735091 (+ 2196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:55.735098 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:55.735277 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:55.735309 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 18:46:55.735406 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 18:46:55.735543 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17531,"mutex_wait_us":146,"spinlock_wait_cycles":12928,"thread_start_us":17673,"threads_started":1}
I20260810 18:46:55.737051 1343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:55.708123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:36229 (local address 127.0.0.1:40538)
0810 18:46:55.724743 (+ 16620us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:55.724803 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:55.725014 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:55.729200 (+ 4186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:55.729205 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:55.729234 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:46:55.729591 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:55.729604 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:55.731681 (+ 2077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:55.731686 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:55.733953 (+ 2267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:55.733970 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:55.736169 (+ 2199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:55.736175 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:55.736362 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:55.736405 (+ 43us) client_negotiation.cc:769] Sending connection context
0810 18:46:55.736505 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 18:46:55.736635 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16341,"mutex_wait_us":8,"thread_start_us":16154,"threads_started":1}
I20260810 18:46:55.737464 1342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:55.708191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40538 (local address 127.4.228.195:36229)
0810 18:46:55.728613 (+ 20422us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:55.728622 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:55.728655 (+ 33us) server_negotiation.cc:407] Connection header received
0810 18:46:55.728746 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:55.728754 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:55.728981 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:55.729173 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:46:55.729789 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:55.731486 (+ 1697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:55.734158 (+ 2672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:55.735674 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:55.735803 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:55.736011 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:55.736927 (+ 916us) server_negotiation.cc:299] Negotiation successful
0810 18:46:55.737108 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20235,"thread_start_us":20084,"threads_started":1}
I20260810 18:46:55.740458 818 tablet_service.cc:1467] Processing CreateTablet for tablet 421123f60fb747ea83cfcb099a1eb544 (DEFAULT_TABLE table=test-table [id=05cb207f98b74451ba3389ce05bf268b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:46:55.740792 1003 tablet_service.cc:1467] Processing CreateTablet for tablet 421123f60fb747ea83cfcb099a1eb544 (DEFAULT_TABLE table=test-table [id=05cb207f98b74451ba3389ce05bf268b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:46:55.741899 818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 421123f60fb747ea83cfcb099a1eb544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:46:55.742223 1003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 421123f60fb747ea83cfcb099a1eb544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:46:55.742178 1209 tablet_service.cc:1467] Processing CreateTablet for tablet 421123f60fb747ea83cfcb099a1eb544 (DEFAULT_TABLE table=test-table [id=05cb207f98b74451ba3389ce05bf268b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:46:55.743659 1209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 421123f60fb747ea83cfcb099a1eb544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:46:55.754791 1349 tablet_bootstrap.cc:492] T 421123f60fb747ea83cfcb099a1eb544 P 40564b1a8574472baf18aad60dcd95bc: Bootstrap starting.
I20260810 18:46:55.758582 1349 tablet_bootstrap.cc:654] T 421123f60fb747ea83cfcb099a1eb544 P 40564b1a8574472baf18aad60dcd95bc: Neither blocks nor log segments found. Creating new log.
I20260810 18:46:55.759109 1352 tablet_bootstrap.cc:492] T 421123f60fb747ea83cfcb099a1eb544 P 8cfc6df8c120471f8a48acb1d34263bc: Bootstrap starting.
I20260810 18:46:55.760044 1349 log.cc:824] T 421123f60fb747ea83cfcb099a1eb544 P 40564b1a8574472baf18aad60dcd95bc: Log is configured to *not* fsync() on all Append() calls
I20260810 18:46:55.760064 1353 tablet_bootstrap.cc:492] T 421123f60fb747ea83cfcb099a1eb544 P d8fed23c259e43ccb6b3e46fed3c4b01: Bootstrap starting.
I20260810 18:46:55.762861 1352 tablet_bootstrap.cc:654] T 421123f60fb747ea83cfcb099a1eb544 P 8cfc6df8c120471f8a48acb1d34263bc: Neither blocks nor log segments found. Creating new log.
I20260810 18:46:55.763587 1353 tablet_bootstrap.cc:654] T 421123f60fb747ea83cfcb099a1eb544 P d8fed23c259e43ccb6b3e46fed3c4b01: Neither blocks nor log segments found. Creating new log.
I20260810 18:46:55.764178 1352 log.cc:824] T 421123f60fb747ea83cfcb099a1eb544 P 8cfc6df8c120471f8a48acb1d34263bc: Log is configured to *not* fsync() on all Append() calls
I20260810 18:46:55.764397 1349 tablet_bootstrap.cc:492] T 421123f60fb747ea83cfcb099a1eb544 P 40564b1a8574472baf18aad60dcd95bc: No bootstrap required, opened a new log
I20260810 18:46:55.764657 1353 log.cc:824] T 421123f60fb747ea83cfcb099a1eb544 P d8fed23c259e43ccb6b3e46fed3c4b01: Log is configured to *not* fsync() on all Append() calls
I20260810 18:46:55.764695 1349 ts_tablet_manager.cc:1397] T 421123f60fb747ea83cfcb099a1eb544 P 40564b1a8574472baf18aad60dcd95bc: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260810 18:46:55.767443 1352 tablet_bootstrap.cc:492] T 421123f60fb747ea83cfcb099a1eb544 P 8cfc6df8c120471f8a48acb1d34263bc: No bootstrap required, opened a new log
I20260810 18:46:55.767716 1352 ts_tablet_manager.cc:1397] T 421123f60fb747ea83cfcb099a1eb544 P 8cfc6df8c120471f8a48acb1d34263bc: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260810 18:46:55.768810 1353 tablet_bootstrap.cc:492] T 421123f60fb747ea83cfcb099a1eb544 P d8fed23c259e43ccb6b3e46fed3c4b01: No bootstrap required, opened a new log
I20260810 18:46:55.769085 1353 ts_tablet_manager.cc:1397] T 421123f60fb747ea83cfcb099a1eb544 P d8fed23c259e43ccb6b3e46fed3c4b01: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 18:46:55.775014 1352 raft_consensus.cc:348] T 421123f60fb747ea83cfcb099a1eb544 P 8cfc6df8c120471f8a48acb1d34263bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8fed23c259e43ccb6b3e46fed3c4b01" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36229 } } peers { permanent_uuid: "8cfc6df8c120471f8a48acb1d34263bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 39491 } } peers { permanent_uuid: "40564b1a8574472baf18aad60dcd95bc" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 46489 } }
I20260810 18:46:55.775442 1352 raft_consensus.cc:374] T 421123f60fb747ea83cfcb099a1eb544 P 8cfc6df8c120471f8a48acb1d34263bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:46:55.775548 1352 raft_consensus.cc:729] T 421123f60fb747ea83cfcb099a1eb544 P 8cfc6df8c120471f8a48acb1d34263bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cfc6df8c120471f8a48acb1d34263bc, State: Initialized, Role: FOLLOWER
I20260810 18:46:55.775996 1349 raft_consensus.cc:348] T 421123f60fb747ea83cfcb099a1eb544 P 40564b1a8574472baf18aad60dcd95bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8fed23c259e43ccb6b3e46fed3c4b01" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36229 } } peers { permanent_uuid: "8cfc6df8c120471f8a48acb1d34263bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 39491 } } peers { permanent_uuid: "40564b1a8574472baf18aad60dcd95bc" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 46489 } }
I20260810 18:46:55.776226 1352 consensus_queue.cc:260] T 421123f60fb747ea83cfcb099a1eb544 P 8cfc6df8c120471f8a48acb1d34263bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8fed23c259e43ccb6b3e46fed3c4b01" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36229 } } peers { permanent_uuid: "8cfc6df8c120471f8a48acb1d34263bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 39491 } } peers { permanent_uuid: "40564b1a8574472baf18aad60dcd95bc" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 46489 } }
I20260810 18:46:55.776438 1349 raft_consensus.cc:374] T 421123f60fb747ea83cfcb099a1eb544 P 40564b1a8574472baf18aad60dcd95bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:46:55.776576 1349 raft_consensus.cc:729] T 421123f60fb747ea83cfcb099a1eb544 P 40564b1a8574472baf18aad60dcd95bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40564b1a8574472baf18aad60dcd95bc, State: Initialized, Role: FOLLOWER
I20260810 18:46:55.777321 1349 consensus_queue.cc:260] T 421123f60fb747ea83cfcb099a1eb544 P 40564b1a8574472baf18aad60dcd95bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8fed23c259e43ccb6b3e46fed3c4b01" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36229 } } peers { permanent_uuid: "8cfc6df8c120471f8a48acb1d34263bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 39491 } } peers { permanent_uuid: "40564b1a8574472baf18aad60dcd95bc" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 46489 } }
I20260810 18:46:55.778621 1103 heartbeater.cc:503] Master 127.4.228.254:33263 was elected leader, sending a full tablet report...
W20260810 18:46:55.780344 912 tablet.cc:2369] T 421123f60fb747ea83cfcb099a1eb544 P 40564b1a8574472baf18aad60dcd95bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:46:55.780647 1353 raft_consensus.cc:348] T 421123f60fb747ea83cfcb099a1eb544 P d8fed23c259e43ccb6b3e46fed3c4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8fed23c259e43ccb6b3e46fed3c4b01" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36229 } } peers { permanent_uuid: "8cfc6df8c120471f8a48acb1d34263bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 39491 } } peers { permanent_uuid: "40564b1a8574472baf18aad60dcd95bc" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 46489 } }
I20260810 18:46:55.781143 1353 raft_consensus.cc:374] T 421123f60fb747ea83cfcb099a1eb544 P d8fed23c259e43ccb6b3e46fed3c4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:46:55.781279 1353 raft_consensus.cc:729] T 421123f60fb747ea83cfcb099a1eb544 P d8fed23c259e43ccb6b3e46fed3c4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8fed23c259e43ccb6b3e46fed3c4b01, State: Initialized, Role: FOLLOWER
I20260810 18:46:55.781744 1352 ts_tablet_manager.cc:1428] T 421123f60fb747ea83cfcb099a1eb544 P 8cfc6df8c120471f8a48acb1d34263bc: Time spent starting tablet: real 0.014s user 0.011s sys 0.001s
I20260810 18:46:55.781966 1353 consensus_queue.cc:260] T 421123f60fb747ea83cfcb099a1eb544 P d8fed23c259e43ccb6b3e46fed3c4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8fed23c259e43ccb6b3e46fed3c4b01" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36229 } } peers { permanent_uuid: "8cfc6df8c120471f8a48acb1d34263bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 39491 } } peers { permanent_uuid: "40564b1a8574472baf18aad60dcd95bc" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 46489 } }
I20260810 18:46:55.782637 1349 ts_tablet_manager.cc:1428] T 421123f60fb747ea83cfcb099a1eb544 P 40564b1a8574472baf18aad60dcd95bc: Time spent starting tablet: real 0.018s user 0.015s sys 0.003s
I20260810 18:46:55.783345 1320 heartbeater.cc:503] Master 127.4.228.254:33263 was elected leader, sending a full tablet report...
I20260810 18:46:55.784054 1353 ts_tablet_manager.cc:1428] T 421123f60fb747ea83cfcb099a1eb544 P d8fed23c259e43ccb6b3e46fed3c4b01: Time spent starting tablet: real 0.015s user 0.009s sys 0.007s
I20260810 18:46:55.817727 1360 raft_consensus.cc:482] T 421123f60fb747ea83cfcb099a1eb544 P 40564b1a8574472baf18aad60dcd95bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:46:55.818120 1360 raft_consensus.cc:504] T 421123f60fb747ea83cfcb099a1eb544 P 40564b1a8574472baf18aad60dcd95bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8fed23c259e43ccb6b3e46fed3c4b01" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36229 } } peers { permanent_uuid: "8cfc6df8c120471f8a48acb1d34263bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 39491 } } peers { permanent_uuid: "40564b1a8574472baf18aad60dcd95bc" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 46489 } }
I20260810 18:46:55.819919 1360 leader_election.cc:288] T 421123f60fb747ea83cfcb099a1eb544 P 40564b1a8574472baf18aad60dcd95bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8fed23c259e43ccb6b3e46fed3c4b01 (127.4.228.195:36229), 8cfc6df8c120471f8a48acb1d34263bc (127.4.228.194:39491)
I20260810 18:46:55.827006 1342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:55.820700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:49015 (local address 127.4.228.195:36229)
0810 18:46:55.820910 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:55.820917 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:55.821079 (+ 162us) server_negotiation.cc:407] Connection header received
0810 18:46:55.821376 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:55.821382 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:55.821444 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:55.821562 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:46:55.822102 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:55.822993 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:55.825017 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:55.825846 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:55.825977 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:55.826083 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:55.826452 (+ 369us) server_negotiation.cc:299] Negotiation successful
0810 18:46:55.826645 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 18:46:55.827004 1363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:55.820142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:36229 (local address 127.4.228.193:49015)
0810 18:46:55.820974 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:55.820996 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:55.821209 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:55.821620 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:55.821624 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:55.821659 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:46:55.821949 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:55.821957 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:55.823132 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:55.823141 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:55.824832 (+ 1691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:55.824845 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:55.826083 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:55.826089 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:55.826289 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:55.826314 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 18:46:55.826417 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 18:46:55.826570 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":698,"mutex_wait_us":29,"thread_start_us":782,"threads_started":1}
I20260810 18:46:55.827113 1346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:55.820616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:56441 (local address 127.4.228.194:39491)
0810 18:46:55.820915 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:55.820924 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:55.821513 (+ 589us) server_negotiation.cc:407] Connection header received
0810 18:46:55.821686 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:55.821690 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:55.821759 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:55.821860 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:46:55.822510 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:55.823215 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:55.825065 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:55.825933 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:55.826098 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:55.826176 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:55.826712 (+ 536us) server_negotiation.cc:299] Negotiation successful
0810 18:46:55.826862 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260810 18:46:55.827781 1365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:55.820406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:39491 (local address 127.4.228.193:56441)
0810 18:46:55.821394 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:55.821409 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:55.821519 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:55.821883 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:55.821887 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:55.821920 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:46:55.822305 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:55.822320 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:55.823359 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:55.823364 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:55.824926 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:55.824946 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:55.826259 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:55.826264 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:55.826418 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:55.826451 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 18:46:55.827316 (+ 865us) client_negotiation.cc:240] Negotiation successful
0810 18:46:55.827441 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":880,"mutex_wait_us":1,"thread_start_us":699,"threads_started":1}
W20260810 18:46:55.828656 1104 tablet.cc:2369] T 421123f60fb747ea83cfcb099a1eb544 P 8cfc6df8c120471f8a48acb1d34263bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:46:55.828491 1238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "421123f60fb747ea83cfcb099a1eb544" candidate_uuid: "40564b1a8574472baf18aad60dcd95bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8fed23c259e43ccb6b3e46fed3c4b01" is_pre_election: true
I20260810 18:46:55.829141 1238 raft_consensus.cc:2454] T 421123f60fb747ea83cfcb099a1eb544 P d8fed23c259e43ccb6b3e46fed3c4b01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40564b1a8574472baf18aad60dcd95bc in term 0.
I20260810 18:46:55.829034 1037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "421123f60fb747ea83cfcb099a1eb544" candidate_uuid: "40564b1a8574472baf18aad60dcd95bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cfc6df8c120471f8a48acb1d34263bc" is_pre_election: true
I20260810 18:46:55.829638 1037 raft_consensus.cc:2454] T 421123f60fb747ea83cfcb099a1eb544 P 8cfc6df8c120471f8a48acb1d34263bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40564b1a8574472baf18aad60dcd95bc in term 0.
I20260810 18:46:55.829885 749 leader_election.cc:302] T 421123f60fb747ea83cfcb099a1eb544 P 40564b1a8574472baf18aad60dcd95bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40564b1a8574472baf18aad60dcd95bc, d8fed23c259e43ccb6b3e46fed3c4b01; no voters:
I20260810 18:46:55.830379 1360 raft_consensus.cc:2781] T 421123f60fb747ea83cfcb099a1eb544 P 40564b1a8574472baf18aad60dcd95bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:46:55.830575 1360 raft_consensus.cc:482] T 421123f60fb747ea83cfcb099a1eb544 P 40564b1a8574472baf18aad60dcd95bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:46:55.830709 1360 raft_consensus.cc:3037] T 421123f60fb747ea83cfcb099a1eb544 P 40564b1a8574472baf18aad60dcd95bc [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:46:55.832989 1360 raft_consensus.cc:504] T 421123f60fb747ea83cfcb099a1eb544 P 40564b1a8574472baf18aad60dcd95bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8fed23c259e43ccb6b3e46fed3c4b01" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36229 } } peers { permanent_uuid: "8cfc6df8c120471f8a48acb1d34263bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 39491 } } peers { permanent_uuid: "40564b1a8574472baf18aad60dcd95bc" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 46489 } }
I20260810 18:46:55.833657 1360 leader_election.cc:288] T 421123f60fb747ea83cfcb099a1eb544 P 40564b1a8574472baf18aad60dcd95bc [CANDIDATE]: Term 1 election: Requested vote from peers d8fed23c259e43ccb6b3e46fed3c4b01 (127.4.228.195:36229), 8cfc6df8c120471f8a48acb1d34263bc (127.4.228.194:39491)
I20260810 18:46:55.834187 1238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "421123f60fb747ea83cfcb099a1eb544" candidate_uuid: "40564b1a8574472baf18aad60dcd95bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8fed23c259e43ccb6b3e46fed3c4b01"
I20260810 18:46:55.834178 1037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "421123f60fb747ea83cfcb099a1eb544" candidate_uuid: "40564b1a8574472baf18aad60dcd95bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cfc6df8c120471f8a48acb1d34263bc"
I20260810 18:46:55.834404 1037 raft_consensus.cc:3037] T 421123f60fb747ea83cfcb099a1eb544 P 8cfc6df8c120471f8a48acb1d34263bc [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:46:55.834400 1238 raft_consensus.cc:3037] T 421123f60fb747ea83cfcb099a1eb544 P d8fed23c259e43ccb6b3e46fed3c4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:46:55.837702 1238 raft_consensus.cc:2454] T 421123f60fb747ea83cfcb099a1eb544 P d8fed23c259e43ccb6b3e46fed3c4b01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40564b1a8574472baf18aad60dcd95bc in term 1.
I20260810 18:46:55.837702 1037 raft_consensus.cc:2454] T 421123f60fb747ea83cfcb099a1eb544 P 8cfc6df8c120471f8a48acb1d34263bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40564b1a8574472baf18aad60dcd95bc in term 1.
I20260810 18:46:55.838120 751 leader_election.cc:302] T 421123f60fb747ea83cfcb099a1eb544 P 40564b1a8574472baf18aad60dcd95bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40564b1a8574472baf18aad60dcd95bc, 8cfc6df8c120471f8a48acb1d34263bc; no voters:
I20260810 18:46:55.838526 1360 raft_consensus.cc:2781] T 421123f60fb747ea83cfcb099a1eb544 P 40564b1a8574472baf18aad60dcd95bc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:46:55.839005 1360 raft_consensus.cc:686] T 421123f60fb747ea83cfcb099a1eb544 P 40564b1a8574472baf18aad60dcd95bc [term 1 LEADER]: Becoming Leader. State: Replica: 40564b1a8574472baf18aad60dcd95bc, State: Running, Role: LEADER
I20260810 18:46:55.839494 1360 consensus_queue.cc:237] T 421123f60fb747ea83cfcb099a1eb544 P 40564b1a8574472baf18aad60dcd95bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8fed23c259e43ccb6b3e46fed3c4b01" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36229 } } peers { permanent_uuid: "8cfc6df8c120471f8a48acb1d34263bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 39491 } } peers { permanent_uuid: "40564b1a8574472baf18aad60dcd95bc" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 46489 } }
I20260810 18:46:55.844432 630 catalog_manager.cc:5495] T 421123f60fb747ea83cfcb099a1eb544 P 40564b1a8574472baf18aad60dcd95bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 40564b1a8574472baf18aad60dcd95bc (127.4.228.193). New cstate: current_term: 1 leader_uuid: "40564b1a8574472baf18aad60dcd95bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8fed23c259e43ccb6b3e46fed3c4b01" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8cfc6df8c120471f8a48acb1d34263bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 39491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40564b1a8574472baf18aad60dcd95bc" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 46489 } health_report { overall_health: HEALTHY } } }
I20260810 18:46:55.869791 1348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:55.862070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40324 (local address 127.4.228.193:46489)
0810 18:46:55.862298 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:55.862302 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:55.862320 (+ 18us) server_negotiation.cc:407] Connection header received
0810 18:46:55.862400 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:55.862403 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:55.862550 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:55.862729 (+ 179us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:46:55.863255 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:55.864782 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:55.867100 (+ 2318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:55.867736 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:55.867848 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:55.868581 (+ 733us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:46:55.869135 (+ 554us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:46:55.869279 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:55.869412 (+ 133us) server_negotiation.cc:299] Negotiation successful
0810 18:46:55.869487 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 18:46:55.878562 1238 raft_consensus.cc:1261] T 421123f60fb747ea83cfcb099a1eb544 P d8fed23c259e43ccb6b3e46fed3c4b01 [term 1 FOLLOWER]: Refusing update from remote peer 40564b1a8574472baf18aad60dcd95bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:46:55.878818 1037 raft_consensus.cc:1261] T 421123f60fb747ea83cfcb099a1eb544 P 8cfc6df8c120471f8a48acb1d34263bc [term 1 FOLLOWER]: Refusing update from remote peer 40564b1a8574472baf18aad60dcd95bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:46:55.879747 1366 consensus_queue.cc:1035] T 421123f60fb747ea83cfcb099a1eb544 P 40564b1a8574472baf18aad60dcd95bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8fed23c259e43ccb6b3e46fed3c4b01" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 36229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:46:55.880182 1360 consensus_queue.cc:1035] T 421123f60fb747ea83cfcb099a1eb544 P 40564b1a8574472baf18aad60dcd95bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "8cfc6df8c120471f8a48acb1d34263bc" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 39491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 18:46:55.883384 1321 tablet.cc:2369] T 421123f60fb747ea83cfcb099a1eb544 P d8fed23c259e43ccb6b3e46fed3c4b01: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:46:55.893150 1371 mvcc.cc:205] Tried to move back new op lower bound from 7317043674627682304 to 7317043674482155520. Current Snapshot: MvccSnapshot[applied={T|T < 7317043674627682304}]
I20260810 18:46:55.894651 1373 mvcc.cc:205] Tried to move back new op lower bound from 7317043674627682304 to 7317043674482155520. Current Snapshot: MvccSnapshot[applied={T|T < 7317043674627682304}]
I20260810 18:46:55.907956 630 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:46:55.908294 630 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:46:55.911975 630 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 44da3965af404d018912552edfcf787a: Sending DeleteTablet for 3 replicas of tablet 421123f60fb747ea83cfcb099a1eb544
I20260810 18:46:55.913225 1209 tablet_service.cc:1514] Processing DeleteTablet for tablet 421123f60fb747ea83cfcb099a1eb544 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:46:55 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:40538
I20260810 18:46:55.913268 1003 tablet_service.cc:1514] Processing DeleteTablet for tablet 421123f60fb747ea83cfcb099a1eb544 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:46:55 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:45636
I20260810 18:46:55.914881 818 tablet_service.cc:1514] Processing DeleteTablet for tablet 421123f60fb747ea83cfcb099a1eb544 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:46:55 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:40322
I20260810 18:46:55.916060 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 729
I20260810 18:46:55.924590 1396 tablet_replica.cc:330] T 421123f60fb747ea83cfcb099a1eb544 P d8fed23c259e43ccb6b3e46fed3c4b01: stopping tablet replica
I20260810 18:46:55.925299 1396 raft_consensus.cc:2229] T 421123f60fb747ea83cfcb099a1eb544 P d8fed23c259e43ccb6b3e46fed3c4b01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:46:55.925974 1396 raft_consensus.cc:2258] T 421123f60fb747ea83cfcb099a1eb544 P d8fed23c259e43ccb6b3e46fed3c4b01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:46:55.926981 1397 tablet_replica.cc:330] T 421123f60fb747ea83cfcb099a1eb544 P 8cfc6df8c120471f8a48acb1d34263bc: stopping tablet replica
I20260810 18:46:55.927515 1397 raft_consensus.cc:2229] T 421123f60fb747ea83cfcb099a1eb544 P 8cfc6df8c120471f8a48acb1d34263bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:46:55.928118 1397 raft_consensus.cc:2258] T 421123f60fb747ea83cfcb099a1eb544 P 8cfc6df8c120471f8a48acb1d34263bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:46:55.929209 1396 ts_tablet_manager.cc:1905] T 421123f60fb747ea83cfcb099a1eb544 P d8fed23c259e43ccb6b3e46fed3c4b01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:46:55.930498 1397 ts_tablet_manager.cc:1905] T 421123f60fb747ea83cfcb099a1eb544 P 8cfc6df8c120471f8a48acb1d34263bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:46:55.935786 1401 tablet_replica.cc:330] T 421123f60fb747ea83cfcb099a1eb544 P 40564b1a8574472baf18aad60dcd95bc: stopping tablet replica
I20260810 18:46:55.936281 1401 raft_consensus.cc:2229] T 421123f60fb747ea83cfcb099a1eb544 P 40564b1a8574472baf18aad60dcd95bc [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:46:55.937071 1401 raft_consensus.cc:2258] T 421123f60fb747ea83cfcb099a1eb544 P 40564b1a8574472baf18aad60dcd95bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:46:55.938095 1397 ts_tablet_manager.cc:1918] T 421123f60fb747ea83cfcb099a1eb544 P 8cfc6df8c120471f8a48acb1d34263bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:46:55.938418 1397 log.cc:1196] T 421123f60fb747ea83cfcb099a1eb544 P 8cfc6df8c120471f8a48acb1d34263bc: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-1/wal/wals/421123f60fb747ea83cfcb099a1eb544
I20260810 18:46:55.939111 1397 ts_tablet_manager.cc:1939] T 421123f60fb747ea83cfcb099a1eb544 P 8cfc6df8c120471f8a48acb1d34263bc: Deleting consensus metadata
I20260810 18:46:55.940416 1396 ts_tablet_manager.cc:1918] T 421123f60fb747ea83cfcb099a1eb544 P d8fed23c259e43ccb6b3e46fed3c4b01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:46:55.940653 1396 log.cc:1196] T 421123f60fb747ea83cfcb099a1eb544 P d8fed23c259e43ccb6b3e46fed3c4b01: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-2/wal/wals/421123f60fb747ea83cfcb099a1eb544
I20260810 18:46:55.940979 1401 ts_tablet_manager.cc:1905] T 421123f60fb747ea83cfcb099a1eb544 P 40564b1a8574472baf18aad60dcd95bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:46:55.941282 1396 ts_tablet_manager.cc:1939] T 421123f60fb747ea83cfcb099a1eb544 P d8fed23c259e43ccb6b3e46fed3c4b01: Deleting consensus metadata
I20260810 18:46:55.941354 610 catalog_manager.cc:4841] TS 8cfc6df8c120471f8a48acb1d34263bc (127.4.228.194:39491): tablet 421123f60fb747ea83cfcb099a1eb544 (table test-table [id=05cb207f98b74451ba3389ce05bf268b]) successfully deleted
I20260810 18:46:55.943135 607 catalog_manager.cc:4841] TS d8fed23c259e43ccb6b3e46fed3c4b01 (127.4.228.195:36229): tablet 421123f60fb747ea83cfcb099a1eb544 (table test-table [id=05cb207f98b74451ba3389ce05bf268b]) successfully deleted
Aug 10 18:46:55 dist-test-slave-7kzw krb5kdc[560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.193@KRBTEST.COM
I20260810 18:46:55.949970 1401 ts_tablet_manager.cc:1918] T 421123f60fb747ea83cfcb099a1eb544 P 40564b1a8574472baf18aad60dcd95bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:46:55.950204 1401 log.cc:1196] T 421123f60fb747ea83cfcb099a1eb544 P 40564b1a8574472baf18aad60dcd95bc: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387267874959-5011-0/minicluster-data/ts-0/wal/wals/421123f60fb747ea83cfcb099a1eb544
I20260810 18:46:55.951031 1401 ts_tablet_manager.cc:1939] T 421123f60fb747ea83cfcb099a1eb544 P 40564b1a8574472baf18aad60dcd95bc: Deleting consensus metadata
I20260810 18:46:55.953348 604 catalog_manager.cc:4841] TS 40564b1a8574472baf18aad60dcd95bc (127.4.228.193:46489): tablet 421123f60fb747ea83cfcb099a1eb544 (table test-table [id=05cb207f98b74451ba3389ce05bf268b]) successfully deleted
I20260810 18:46:55.956151 1348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:55.920681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40334 (local address 127.4.228.193:46489)
0810 18:46:55.921586 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:55.921596 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:55.935019 (+ 13423us) server_negotiation.cc:407] Connection header received
0810 18:46:55.935284 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:55.935292 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:55.935442 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:55.935566 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:55.937705 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:55.938931 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:55.943233 (+ 4302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:55.943857 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:55.943948 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:55.948546 (+ 4598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:55.948559 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:55.948575 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:46:55.948620 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:55.952629 (+ 4009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:55.953657 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:55.953670 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:55.953674 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:55.953911 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:55.954326 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:55.954335 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:55.954339 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:55.954846 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:55.955201 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:55.955527 (+ 326us) server_negotiation.cc:299] Negotiation successful
0810 18:46:55.955779 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":720}
I20260810 18:46:55.956902 904 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:46:56.059805 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 729
I20260810 18:46:56.095157 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 917
Aug 10 18:46:56 dist-test-slave-7kzw krb5kdc[560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.194@KRBTEST.COM
I20260810 18:46:56.111477 1346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:56.096129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45642 (local address 127.4.228.194:39491)
0810 18:46:56.096319 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:56.096325 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:56.096348 (+ 23us) server_negotiation.cc:407] Connection header received
0810 18:46:56.096462 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:56.096467 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:56.096545 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:56.096699 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:56.098030 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:56.098920 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:56.100274 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:56.100687 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:56.100832 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:56.105612 (+ 4780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:56.105626 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:56.105641 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:46:56.105685 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:56.108720 (+ 3035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:56.109389 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:56.109396 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:56.109398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:56.109521 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:56.109956 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:56.109959 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:56.109961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:56.110344 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:56.110668 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:56.111040 (+ 372us) server_negotiation.cc:299] Negotiation successful
0810 18:46:56.111227 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 18:46:56.112270 1097 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:46:56.202416 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 917
I20260810 18:46:56.230829 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 1110
Aug 10 18:46:56 dist-test-slave-7kzw krb5kdc[560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.195@KRBTEST.COM
I20260810 18:46:56.246848 1342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:56.231653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40544 (local address 127.4.228.195:36229)
0810 18:46:56.231852 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:56.231857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:56.231878 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:46:56.232006 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:56.232011 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:56.232088 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:56.232195 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:56.233514 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:56.234282 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:56.235602 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:56.236011 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:56.236217 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:56.241054 (+ 4837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:56.241071 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:56.241098 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:46:56.241134 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:56.243914 (+ 2780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:56.244621 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:56.244630 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:56.244633 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:56.244778 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:46:56.245172 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:46:56.245177 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:46:56.245180 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:46:56.245553 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:56.245865 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:56.246222 (+ 357us) server_negotiation.cc:299] Negotiation successful
0810 18:46:56.246488 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 18:46:56.247599 1311 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:46:56.337185 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 1110
I20260810 18:46:56.363998 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 578
I20260810 18:46:56.365290 683 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:46:56.477804 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 578
2026-08-10T18:46:56Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4849 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-10T18:46:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:46:56Z Disabled control of system clock
I20260810 18:46:56.580947 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:35143
--webserver_interface=127.4.228.254
--webserver_port=0
--builtin_ntp_servers=127.4.228.212:41465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:35143
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 18:46:56.943451 1412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:46:56.943779 1412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:46:56.943922 1412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:46:56.951303 1412 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:46:56.951424 1412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:46:56.951463 1412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:46:56.951574 1412 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:46:56.951643 1412 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:46:56.961189 1412 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:41465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:35143
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:35143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:46:56.962669 1412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:46:56.964587 1412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:46:56.978216 1418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:56.978305 1422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:56.978278 1419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:46:56.979321 1412 server_base.cc:1034] running on GCE node
I20260810 18:46:56.980304 1412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:46:56.982322 1412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:46:56.983596 1412 hybrid_clock.cc:648] HybridClock initialized: now 1786387616983567 us; error 55 us; skew 500 ppm
I20260810 18:46:56.984083 1412 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:46:56.986987 1412 webserver.cc:460] Webserver started at http://127.4.228.254:42957/ using document root <none> and password file <none>
I20260810 18:46:56.987828 1412 fs_manager.cc:359] Metadata directory not provided
I20260810 18:46:56.987977 1412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:46:56.988341 1412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:46:56.991628 1412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "134e5cc9d98843ce8911fd144763de39"
format_stamp: "Formatted at 2026-08-10 18:46:56 on dist-test-slave-7kzw"
server_key: "a15019c6d496c6e37fa2b030d8803979"
server_key_iv: "c975e331773dfa0b9c14f4664bf16b2b"
server_key_version: "encryptionkey@0"
I20260810 18:46:56.992528 1412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "134e5cc9d98843ce8911fd144763de39"
format_stamp: "Formatted at 2026-08-10 18:46:56 on dist-test-slave-7kzw"
server_key: "a15019c6d496c6e37fa2b030d8803979"
server_key_iv: "c975e331773dfa0b9c14f4664bf16b2b"
server_key_version: "encryptionkey@0"
I20260810 18:46:56.999114 1412 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260810 18:46:57.003016 1435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:57.004771 1412 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:46:57.005050 1412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "134e5cc9d98843ce8911fd144763de39"
format_stamp: "Formatted at 2026-08-10 18:46:56 on dist-test-slave-7kzw"
server_key: "a15019c6d496c6e37fa2b030d8803979"
server_key_iv: "c975e331773dfa0b9c14f4664bf16b2b"
server_key_version: "encryptionkey@0"
I20260810 18:46:57.005330 1412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:46:57.025743 1412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:46:57.026906 1412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:46:57.027451 1412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:46:57.049803 1412 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:35143
I20260810 18:46:57.049823 1506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:35143 every 8 connection(s)
I20260810 18:46:57.051311 1412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:46:57.053853 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 1412
I20260810 18:46:57.054050 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:46:57.054597 5011 external_mini_cluster.cc:1428] Setting key 8b7a33ecfebcecc955889a1af2aa1353
I20260810 18:46:57.055908 1507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:46:57.069269 1510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:57.056699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33564 (local address 127.4.228.254:35143)
0810 18:46:57.057781 (+ 1082us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:57.057792 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:57.057836 (+ 44us) server_negotiation.cc:407] Connection header received
0810 18:46:57.059458 (+ 1622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:57.059481 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:57.059973 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:57.060330 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:57.061001 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:57.062389 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:57.064442 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:57.065137 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:57.065277 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:57.066658 (+ 1381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:57.066671 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:57.066685 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:46:57.067305 (+ 620us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:57.067389 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:46:57.067411 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:57.067518 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:57.067852 (+ 334us) server_negotiation.cc:299] Negotiation successful
0810 18:46:57.068273 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":401,"threads_started":1}
I20260810 18:46:57.070297 1507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 134e5cc9d98843ce8911fd144763de39: Bootstrap starting.
I20260810 18:46:57.073939 1507 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 134e5cc9d98843ce8911fd144763de39: Neither blocks nor log segments found. Creating new log.
I20260810 18:46:57.075248 1507 log.cc:824] T 00000000000000000000000000000000 P 134e5cc9d98843ce8911fd144763de39: Log is configured to *not* fsync() on all Append() calls
I20260810 18:46:57.078131 1507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 134e5cc9d98843ce8911fd144763de39: No bootstrap required, opened a new log
I20260810 18:46:57.085997 1507 raft_consensus.cc:348] T 00000000000000000000000000000000 P 134e5cc9d98843ce8911fd144763de39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "134e5cc9d98843ce8911fd144763de39" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35143 } }
I20260810 18:46:57.086474 1507 raft_consensus.cc:374] T 00000000000000000000000000000000 P 134e5cc9d98843ce8911fd144763de39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:46:57.086624 1507 raft_consensus.cc:729] T 00000000000000000000000000000000 P 134e5cc9d98843ce8911fd144763de39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 134e5cc9d98843ce8911fd144763de39, State: Initialized, Role: FOLLOWER
I20260810 18:46:57.087307 1507 consensus_queue.cc:260] T 00000000000000000000000000000000 P 134e5cc9d98843ce8911fd144763de39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "134e5cc9d98843ce8911fd144763de39" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35143 } }
I20260810 18:46:57.087568 1507 raft_consensus.cc:388] T 00000000000000000000000000000000 P 134e5cc9d98843ce8911fd144763de39 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:46:57.087733 1507 raft_consensus.cc:482] T 00000000000000000000000000000000 P 134e5cc9d98843ce8911fd144763de39 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:46:57.087939 1507 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 134e5cc9d98843ce8911fd144763de39 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:46:57.090234 1507 raft_consensus.cc:504] T 00000000000000000000000000000000 P 134e5cc9d98843ce8911fd144763de39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "134e5cc9d98843ce8911fd144763de39" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35143 } }
I20260810 18:46:57.090821 1507 leader_election.cc:302] T 00000000000000000000000000000000 P 134e5cc9d98843ce8911fd144763de39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 134e5cc9d98843ce8911fd144763de39; no voters:
I20260810 18:46:57.091531 1507 leader_election.cc:288] T 00000000000000000000000000000000 P 134e5cc9d98843ce8911fd144763de39 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:46:57.091802 1514 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 134e5cc9d98843ce8911fd144763de39 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:46:57.092701 1514 raft_consensus.cc:686] T 00000000000000000000000000000000 P 134e5cc9d98843ce8911fd144763de39 [term 1 LEADER]: Becoming Leader. State: Replica: 134e5cc9d98843ce8911fd144763de39, State: Running, Role: LEADER
I20260810 18:46:57.093329 1514 consensus_queue.cc:237] T 00000000000000000000000000000000 P 134e5cc9d98843ce8911fd144763de39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "134e5cc9d98843ce8911fd144763de39" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35143 } }
I20260810 18:46:57.093887 1507 sys_catalog.cc:564] T 00000000000000000000000000000000 P 134e5cc9d98843ce8911fd144763de39 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:46:57.096721 1518 sys_catalog.cc:455] T 00000000000000000000000000000000 P 134e5cc9d98843ce8911fd144763de39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 134e5cc9d98843ce8911fd144763de39. Latest consensus state: current_term: 1 leader_uuid: "134e5cc9d98843ce8911fd144763de39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "134e5cc9d98843ce8911fd144763de39" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35143 } } }
I20260810 18:46:57.096750 1516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 134e5cc9d98843ce8911fd144763de39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "134e5cc9d98843ce8911fd144763de39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "134e5cc9d98843ce8911fd144763de39" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 35143 } } }
I20260810 18:46:57.097293 1518 sys_catalog.cc:458] T 00000000000000000000000000000000 P 134e5cc9d98843ce8911fd144763de39 [sys.catalog]: This master's current role is: LEADER
I20260810 18:46:57.097291 1516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 134e5cc9d98843ce8911fd144763de39 [sys.catalog]: This master's current role is: LEADER
I20260810 18:46:57.098438 1526 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:46:57.106047 1526 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:46:57.117982 1526 catalog_manager.cc:1347] Generated new cluster ID: 405c181a070f4b799dac9c679567b1d5
I20260810 18:46:57.118180 1526 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:46:57.126744 1526 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:46:57.128260 1526 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:46:57.144001 1526 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 134e5cc9d98843ce8911fd144763de39: Generated new TSK 0
I20260810 18:46:57.145020 1526 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:46:57.153735 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.193:0
--local_ip_for_outbound_sockets=127.4.228.193
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:35143
--builtin_ntp_servers=127.4.228.212:41465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 18:46:57.513089 1550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:46:57.513388 1550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:46:57.513530 1550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:46:57.520955 1550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:46:57.521063 1550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:46:57.521229 1550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.193
I20260810 18:46:57.530763 1550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:41465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:35143
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.193
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:46:57.532300 1550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:46:57.534220 1550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:46:57.548185 1559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:57.548254 1557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:57.548233 1563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:46:57.549412 1550 server_base.cc:1034] running on GCE node
I20260810 18:46:57.550096 1550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:46:57.551304 1550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:46:57.552758 1550 hybrid_clock.cc:648] HybridClock initialized: now 1786387617552706 us; error 53 us; skew 500 ppm
I20260810 18:46:57.553141 1550 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:46:57.555789 1550 webserver.cc:460] Webserver started at http://127.4.228.193:44765/ using document root <none> and password file <none>
I20260810 18:46:57.556547 1550 fs_manager.cc:359] Metadata directory not provided
I20260810 18:46:57.556697 1550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:46:57.557060 1550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:46:57.560124 1550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-0/data/instance:
uuid: "957f19c319f04f439dff1c455af3da1a"
format_stamp: "Formatted at 2026-08-10 18:46:57 on dist-test-slave-7kzw"
server_key: "9996df009e8463a274418d2aa13eb448"
server_key_iv: "aaaa4f815b88455ba92af39f8a481e56"
server_key_version: "encryptionkey@0"
I20260810 18:46:57.560956 1550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance:
uuid: "957f19c319f04f439dff1c455af3da1a"
format_stamp: "Formatted at 2026-08-10 18:46:57 on dist-test-slave-7kzw"
server_key: "9996df009e8463a274418d2aa13eb448"
server_key_iv: "aaaa4f815b88455ba92af39f8a481e56"
server_key_version: "encryptionkey@0"
I20260810 18:46:57.567504 1550 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260810 18:46:57.571856 1576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:57.573411 1550 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260810 18:46:57.573575 1550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-0/wal
uuid: "957f19c319f04f439dff1c455af3da1a"
format_stamp: "Formatted at 2026-08-10 18:46:57 on dist-test-slave-7kzw"
server_key: "9996df009e8463a274418d2aa13eb448"
server_key_iv: "aaaa4f815b88455ba92af39f8a481e56"
server_key_version: "encryptionkey@0"
I20260810 18:46:57.573827 1550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:46:57.597333 1550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:46:57.598218 1550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:46:57.598579 1550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:46:57.599640 1550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:46:57.601265 1550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:46:57.601365 1550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:57.601446 1550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:46:57.601508 1550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:57.639298 1550 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.193:42905
I20260810 18:46:57.639510 1745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.193:42905 every 8 connection(s)
I20260810 18:46:57.640820 1550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
I20260810 18:46:57.647188 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 1550
I20260810 18:46:57.647437 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance
I20260810 18:46:57.647982 5011 external_mini_cluster.cc:1428] Setting key b3bcf52ab4ae49885e6ba7008b149e62
I20260810 18:46:57.651091 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.194:0
--local_ip_for_outbound_sockets=127.4.228.194
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:35143
--builtin_ntp_servers=127.4.228.212:41465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 18:46:57.654138 1748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:57.643530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:37629 (local address 127.4.228.254:35143)
0810 18:46:57.644035 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:57.644040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:57.645373 (+ 1333us) server_negotiation.cc:407] Connection header received
0810 18:46:57.647290 (+ 1917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:57.647298 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:57.647397 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:57.647524 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:57.648801 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:57.649533 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:57.651157 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:57.651645 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:57.651785 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:57.653229 (+ 1444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:57.653235 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:57.653238 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:46:57.653256 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:57.653297 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:46:57.653309 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:57.653408 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:57.653653 (+ 245us) server_negotiation.cc:299] Negotiation successful
0810 18:46:57.653752 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":270,"threads_started":1}
I20260810 18:46:57.655224 1749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:57.644216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35143 (local address 127.4.228.193:37629)
0810 18:46:57.645210 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:57.645255 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:57.647004 (+ 1749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:57.647820 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:57.647833 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:57.647959 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:46:57.648626 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:57.648643 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:57.649709 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:57.649713 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:57.651010 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:57.651022 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:57.651881 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:57.651889 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:57.652119 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:57.652896 (+ 777us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:46:57.652899 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:46:57.652937 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:46:57.653017 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:46:57.653410 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:46:57.653425 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:46:57.653434 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 18:46:57.653628 (+ 194us) client_negotiation.cc:240] Negotiation successful
0810 18:46:57.654031 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":323,"threads_started":1}
I20260810 18:46:57.656843 1746 heartbeater.cc:347] Connected to a master server at 127.4.228.254:35143
I20260810 18:46:57.657164 1746 heartbeater.cc:464] Registering TS with master...
I20260810 18:46:57.657930 1746 heartbeater.cc:511] Master 127.4.228.254:35143 requested a full tablet report, sending...
I20260810 18:46:57.660439 1458 ts_manager.cc:194] Registered new tserver with Master: 957f19c319f04f439dff1c455af3da1a (127.4.228.193:42905)
I20260810 18:46:57.662873 1458 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.228.193:37629
W20260810 18:46:58.008595 1751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:46:58.008899 1751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:46:58.009073 1751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:46:58.016538 1751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:46:58.016645 1751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:46:58.016814 1751 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.194
I20260810 18:46:58.025802 1751 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:41465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:35143
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.194
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:46:58.027233 1751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:46:58.029341 1751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:46:58.043325 1758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:58.043292 1759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:58.043462 1761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:46:58.045019 1751 server_base.cc:1034] running on GCE node
I20260810 18:46:58.045656 1751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:46:58.046653 1751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:46:58.047883 1751 hybrid_clock.cc:648] HybridClock initialized: now 1786387618047816 us; error 48 us; skew 500 ppm
I20260810 18:46:58.048233 1751 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:46:58.063139 1751 webserver.cc:460] Webserver started at http://127.4.228.194:34105/ using document root <none> and password file <none>
I20260810 18:46:58.063922 1751 fs_manager.cc:359] Metadata directory not provided
I20260810 18:46:58.064047 1751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:46:58.064411 1751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:46:58.067539 1751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-1/data/instance:
uuid: "66720c194ed0469d96e37857ebdbfbea"
format_stamp: "Formatted at 2026-08-10 18:46:58 on dist-test-slave-7kzw"
server_key: "6cfc0d016f30d73ff40d51ad0320ec64"
server_key_iv: "df2ba147692ecf42182fa93e80816a07"
server_key_version: "encryptionkey@0"
I20260810 18:46:58.068348 1751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance:
uuid: "66720c194ed0469d96e37857ebdbfbea"
format_stamp: "Formatted at 2026-08-10 18:46:58 on dist-test-slave-7kzw"
server_key: "6cfc0d016f30d73ff40d51ad0320ec64"
server_key_iv: "df2ba147692ecf42182fa93e80816a07"
server_key_version: "encryptionkey@0"
I20260810 18:46:58.073606 1751 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 18:46:58.077282 1767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:58.078894 1751 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 18:46:58.079062 1751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-1/wal
uuid: "66720c194ed0469d96e37857ebdbfbea"
format_stamp: "Formatted at 2026-08-10 18:46:58 on dist-test-slave-7kzw"
server_key: "6cfc0d016f30d73ff40d51ad0320ec64"
server_key_iv: "df2ba147692ecf42182fa93e80816a07"
server_key_version: "encryptionkey@0"
I20260810 18:46:58.079329 1751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:46:58.089993 1751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:46:58.090852 1751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:46:58.091164 1751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:46:58.092162 1751 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:46:58.093680 1751 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:46:58.093781 1751 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:58.093892 1751 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:46:58.093967 1751 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:58.129236 1751 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.194:41369
I20260810 18:46:58.129321 1939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.194:41369 every 8 connection(s)
I20260810 18:46:58.130661 1751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
I20260810 18:46:58.138893 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 1751
I20260810 18:46:58.139079 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance
I20260810 18:46:58.139561 5011 external_mini_cluster.cc:1428] Setting key 46d6272b451afd15de277b87290ac64e
I20260810 18:46:58.142393 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.195:0
--local_ip_for_outbound_sockets=127.4.228.195
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:35143
--builtin_ntp_servers=127.4.228.212:41465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 18:46:58.144316 1748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:58.133492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:49151 (local address 127.4.228.254:35143)
0810 18:46:58.133688 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:58.133693 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:58.135216 (+ 1523us) server_negotiation.cc:407] Connection header received
0810 18:46:58.137067 (+ 1851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:58.137073 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:58.137174 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:58.137289 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:58.138365 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:58.139270 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:58.141045 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:58.141606 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:58.141743 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:58.143461 (+ 1718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:58.143470 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:58.143479 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:46:58.143496 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:58.143526 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:46:58.143538 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:58.143638 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:58.143892 (+ 254us) server_negotiation.cc:299] Negotiation successful
0810 18:46:58.143991 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 18:46:58.145380 1943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:58.134034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35143 (local address 127.4.228.194:49151)
0810 18:46:58.135045 (+ 1011us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:58.135092 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:58.136790 (+ 1698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:58.137518 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:58.137526 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:58.137617 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:46:58.138189 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:58.138206 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:58.139431 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:58.139436 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:58.140898 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:58.140910 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:58.141844 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:58.141852 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:58.142074 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:58.143139 (+ 1065us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:46:58.143142 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:46:58.143181 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:46:58.143260 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:46:58.143651 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:46:58.143664 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:46:58.143685 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 18:46:58.143879 (+ 194us) client_negotiation.cc:240] Negotiation successful
0810 18:46:58.144267 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":294,"threads_started":1}
I20260810 18:46:58.146843 1940 heartbeater.cc:347] Connected to a master server at 127.4.228.254:35143
I20260810 18:46:58.147174 1940 heartbeater.cc:464] Registering TS with master...
I20260810 18:46:58.147933 1940 heartbeater.cc:511] Master 127.4.228.254:35143 requested a full tablet report, sending...
I20260810 18:46:58.149374 1458 ts_manager.cc:194] Registered new tserver with Master: 66720c194ed0469d96e37857ebdbfbea (127.4.228.194:41369)
I20260810 18:46:58.150187 1458 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.228.194:49151
W20260810 18:46:58.493242 1944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:46:58.493546 1944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:46:58.493737 1944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:46:58.500972 1944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:46:58.501067 1944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:46:58.501173 1944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.195
I20260810 18:46:58.510859 1944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:41465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:35143
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.195
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:46:58.512174 1944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:46:58.514158 1944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:46:58.527645 1953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:58.527522 1956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:46:58.527608 1954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:46:58.528733 1944 server_base.cc:1034] running on GCE node
I20260810 18:46:58.529496 1944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:46:58.530725 1944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:46:58.531977 1944 hybrid_clock.cc:648] HybridClock initialized: now 1786387618531911 us; error 50 us; skew 500 ppm
I20260810 18:46:58.532292 1944 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:46:58.537796 1944 webserver.cc:460] Webserver started at http://127.4.228.195:40303/ using document root <none> and password file <none>
I20260810 18:46:58.538627 1944 fs_manager.cc:359] Metadata directory not provided
I20260810 18:46:58.538731 1944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:46:58.539016 1944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:46:58.541914 1944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-2/data/instance:
uuid: "d8bd7f2d079e486dacf92ae5fab9f9d8"
format_stamp: "Formatted at 2026-08-10 18:46:58 on dist-test-slave-7kzw"
server_key: "5ab326a1714d2ef2624250c336c89498"
server_key_iv: "683015dfd7263d91f61b90bb0ee4a2ae"
server_key_version: "encryptionkey@0"
I20260810 18:46:58.542771 1944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance:
uuid: "d8bd7f2d079e486dacf92ae5fab9f9d8"
format_stamp: "Formatted at 2026-08-10 18:46:58 on dist-test-slave-7kzw"
server_key: "5ab326a1714d2ef2624250c336c89498"
server_key_iv: "683015dfd7263d91f61b90bb0ee4a2ae"
server_key_version: "encryptionkey@0"
I20260810 18:46:58.548200 1944 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 18:46:58.551834 1965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:58.553328 1944 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:46:58.553478 1944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-2/wal
uuid: "d8bd7f2d079e486dacf92ae5fab9f9d8"
format_stamp: "Formatted at 2026-08-10 18:46:58 on dist-test-slave-7kzw"
server_key: "5ab326a1714d2ef2624250c336c89498"
server_key_iv: "683015dfd7263d91f61b90bb0ee4a2ae"
server_key_version: "encryptionkey@0"
I20260810 18:46:58.553719 1944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:46:58.576006 1944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:46:58.577035 1944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:46:58.577310 1944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:46:58.578409 1944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:46:58.580046 1944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:46:58.580147 1944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:58.580224 1944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:46:58.580279 1944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:46:58.615849 1944 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.195:43451
I20260810 18:46:58.615947 2133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.195:43451 every 8 connection(s)
I20260810 18:46:58.617216 1944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
I20260810 18:46:58.625409 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 1944
I20260810 18:46:58.625608 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance
I20260810 18:46:58.626194 5011 external_mini_cluster.cc:1428] Setting key 70990c8b5b6704d848687ae91ce2beb2
I20260810 18:46:58.631245 1748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:58.620179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:44745 (local address 127.4.228.254:35143)
0810 18:46:58.620380 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:58.620385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:58.621803 (+ 1418us) server_negotiation.cc:407] Connection header received
0810 18:46:58.623680 (+ 1877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:58.623685 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:58.623760 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:58.623843 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:58.624992 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:58.625888 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:58.628110 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:58.628770 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:58.628908 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:58.630422 (+ 1514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:58.630428 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:58.630431 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:46:58.630448 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:58.630532 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:46:58.630549 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:58.630633 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:58.630873 (+ 240us) server_negotiation.cc:299] Negotiation successful
0810 18:46:58.630955 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 18:46:58.632205 2137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:58.620630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:35143 (local address 127.4.228.195:44745)
0810 18:46:58.621620 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:58.621668 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:58.623393 (+ 1725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:58.624093 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:58.624101 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:58.624188 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:46:58.624809 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:58.624826 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:58.626071 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:58.626077 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:58.627947 (+ 1870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:58.627965 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:58.629019 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:58.629025 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:58.629391 (+ 366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:58.630098 (+ 707us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:46:58.630101 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:46:58.630140 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:46:58.630243 (+ 103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:46:58.630647 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:46:58.630655 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:46:58.630661 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 18:46:58.630809 (+ 148us) client_negotiation.cc:240] Negotiation successful
0810 18:46:58.631097 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":309,"threads_started":1}
I20260810 18:46:58.633584 2134 heartbeater.cc:347] Connected to a master server at 127.4.228.254:35143
I20260810 18:46:58.633885 2134 heartbeater.cc:464] Registering TS with master...
I20260810 18:46:58.634693 2134 heartbeater.cc:511] Master 127.4.228.254:35143 requested a full tablet report, sending...
I20260810 18:46:58.636085 1458 ts_manager.cc:194] Registered new tserver with Master: d8bd7f2d079e486dacf92ae5fab9f9d8 (127.4.228.195:43451)
I20260810 18:46:58.636873 1458 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.228.195:44745
I20260810 18:46:58.641928 5011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:46:58.653563 1748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:58.647218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33578 (local address 127.4.228.254:35143)
0810 18:46:58.647431 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:58.647441 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:58.647885 (+ 444us) server_negotiation.cc:407] Connection header received
0810 18:46:58.648096 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:58.648102 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:58.648182 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:58.648286 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:58.648824 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:58.649511 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:58.650994 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:58.651457 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:58.651600 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:58.652810 (+ 1210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:58.652816 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:58.652819 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:46:58.652837 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:58.652866 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:46:58.652878 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:58.653033 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:58.653237 (+ 204us) server_negotiation.cc:299] Negotiation successful
0810 18:46:58.653295 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 18:46:58.657323 1458 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33578:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:46:58.664395 1458 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:46:58.668286 1746 heartbeater.cc:503] Master 127.4.228.254:35143 was elected leader, sending a full tablet report...
I20260810 18:46:58.688148 2151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:58.677259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:42905 (local address 127.0.0.1:60076)
0810 18:46:58.678056 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:58.678109 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:58.678304 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:58.679556 (+ 1252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:58.679568 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:58.679613 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:46:58.680074 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:58.680090 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:58.682683 (+ 2593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:58.682690 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:58.684904 (+ 2214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:58.684923 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:58.686937 (+ 2014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:58.686945 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:58.687293 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:58.687351 (+ 58us) client_negotiation.cc:769] Sending connection context
0810 18:46:58.687509 (+ 158us) client_negotiation.cc:240] Negotiation successful
0810 18:46:58.687720 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"mutex_wait_us":40,"thread_start_us":326,"threads_started":1}
I20260810 18:46:58.688159 2152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:58.677576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60076 (local address 127.4.228.193:42905)
0810 18:46:58.678272 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:58.678281 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:58.678368 (+ 87us) server_negotiation.cc:407] Connection header received
0810 18:46:58.678661 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:58.678670 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:58.678922 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:58.679182 (+ 260us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:46:58.680332 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:58.682430 (+ 2098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:58.685133 (+ 2703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:58.686725 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:58.686871 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:58.687210 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:58.687588 (+ 378us) server_negotiation.cc:299] Negotiation successful
0810 18:46:58.687774 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":345,"threads_started":1}
I20260810 18:46:58.691625 2153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:58.678426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:41369 (local address 127.0.0.1:60008)
0810 18:46:58.679098 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:58.679119 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:58.679345 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:58.683669 (+ 4324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:58.683678 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:58.683703 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:46:58.684331 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:58.684349 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:58.686248 (+ 1899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:58.686254 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:58.688805 (+ 2551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:58.688824 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:58.690795 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:58.690802 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:58.690979 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:58.691013 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 18:46:58.691117 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 18:46:58.691250 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"mutex_wait_us":161,"thread_start_us":349,"threads_started":1}
I20260810 18:46:58.691699 2155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:58.678712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60008 (local address 127.4.228.194:41369)
0810 18:46:58.683099 (+ 4387us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:58.683107 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:58.683136 (+ 29us) server_negotiation.cc:407] Connection header received
0810 18:46:58.683226 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:58.683233 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:58.683431 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:58.684281 (+ 850us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:46:58.684549 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:58.686079 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:58.689023 (+ 2944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:58.690584 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:58.690732 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:58.690978 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:58.691185 (+ 207us) server_negotiation.cc:299] Negotiation successful
0810 18:46:58.691361 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4224,"thread_start_us":4682,"threads_started":1}
I20260810 18:46:58.692864 2157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:58.680024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:43451 (local address 127.0.0.1:51144)
0810 18:46:58.682555 (+ 2531us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:58.682575 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:58.682810 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:58.683890 (+ 1080us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:58.683894 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:58.683920 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:46:58.684454 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:58.684468 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:58.686255 (+ 1787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:58.686263 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:58.688813 (+ 2550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:58.688849 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:58.692087 (+ 3238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:58.692096 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:58.692274 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:58.692305 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 18:46:58.692399 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 18:46:58.692531 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2391,"mutex_wait_us":136,"thread_start_us":2260,"threads_started":1}
I20260810 18:46:58.693001 2156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:58.679536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51144 (local address 127.4.228.195:43451)
0810 18:46:58.680259 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:58.680267 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:58.682677 (+ 2410us) server_negotiation.cc:407] Connection header received
0810 18:46:58.683029 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:58.683038 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:58.683369 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:58.683547 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:46:58.684674 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:58.686079 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:58.689143 (+ 3064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:58.691250 (+ 2107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:58.691378 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:58.691645 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:58.692476 (+ 831us) server_negotiation.cc:299] Negotiation successful
0810 18:46:58.692644 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":603,"threads_started":1}
I20260810 18:46:58.693305 1644 tablet_service.cc:1467] Processing CreateTablet for tablet 304a7a44df824118bf934b6014bb1bf9 (DEFAULT_TABLE table=test-table [id=354b70701fc7494bbd8315f95d88738e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:46:58.695072 1644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 304a7a44df824118bf934b6014bb1bf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:46:58.695255 1842 tablet_service.cc:1467] Processing CreateTablet for tablet 304a7a44df824118bf934b6014bb1bf9 (DEFAULT_TABLE table=test-table [id=354b70701fc7494bbd8315f95d88738e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:46:58.696329 1842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 304a7a44df824118bf934b6014bb1bf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:46:58.696889 2039 tablet_service.cc:1467] Processing CreateTablet for tablet 304a7a44df824118bf934b6014bb1bf9 (DEFAULT_TABLE table=test-table [id=354b70701fc7494bbd8315f95d88738e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:46:58.698310 2039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 304a7a44df824118bf934b6014bb1bf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:46:58.707546 2158 tablet_bootstrap.cc:492] T 304a7a44df824118bf934b6014bb1bf9 P 66720c194ed0469d96e37857ebdbfbea: Bootstrap starting.
I20260810 18:46:58.710548 2159 tablet_bootstrap.cc:492] T 304a7a44df824118bf934b6014bb1bf9 P 957f19c319f04f439dff1c455af3da1a: Bootstrap starting.
I20260810 18:46:58.712142 2158 tablet_bootstrap.cc:654] T 304a7a44df824118bf934b6014bb1bf9 P 66720c194ed0469d96e37857ebdbfbea: Neither blocks nor log segments found. Creating new log.
I20260810 18:46:58.713047 2160 tablet_bootstrap.cc:492] T 304a7a44df824118bf934b6014bb1bf9 P d8bd7f2d079e486dacf92ae5fab9f9d8: Bootstrap starting.
I20260810 18:46:58.713864 2158 log.cc:824] T 304a7a44df824118bf934b6014bb1bf9 P 66720c194ed0469d96e37857ebdbfbea: Log is configured to *not* fsync() on all Append() calls
I20260810 18:46:58.714192 2159 tablet_bootstrap.cc:654] T 304a7a44df824118bf934b6014bb1bf9 P 957f19c319f04f439dff1c455af3da1a: Neither blocks nor log segments found. Creating new log.
I20260810 18:46:58.715777 2159 log.cc:824] T 304a7a44df824118bf934b6014bb1bf9 P 957f19c319f04f439dff1c455af3da1a: Log is configured to *not* fsync() on all Append() calls
I20260810 18:46:58.716588 2160 tablet_bootstrap.cc:654] T 304a7a44df824118bf934b6014bb1bf9 P d8bd7f2d079e486dacf92ae5fab9f9d8: Neither blocks nor log segments found. Creating new log.
I20260810 18:46:58.717391 2158 tablet_bootstrap.cc:492] T 304a7a44df824118bf934b6014bb1bf9 P 66720c194ed0469d96e37857ebdbfbea: No bootstrap required, opened a new log
I20260810 18:46:58.717667 2158 ts_tablet_manager.cc:1397] T 304a7a44df824118bf934b6014bb1bf9 P 66720c194ed0469d96e37857ebdbfbea: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260810 18:46:58.717805 2160 log.cc:824] T 304a7a44df824118bf934b6014bb1bf9 P d8bd7f2d079e486dacf92ae5fab9f9d8: Log is configured to *not* fsync() on all Append() calls
I20260810 18:46:58.719069 2159 tablet_bootstrap.cc:492] T 304a7a44df824118bf934b6014bb1bf9 P 957f19c319f04f439dff1c455af3da1a: No bootstrap required, opened a new log
I20260810 18:46:58.719333 2159 ts_tablet_manager.cc:1397] T 304a7a44df824118bf934b6014bb1bf9 P 957f19c319f04f439dff1c455af3da1a: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.005s
I20260810 18:46:58.720381 2160 tablet_bootstrap.cc:492] T 304a7a44df824118bf934b6014bb1bf9 P d8bd7f2d079e486dacf92ae5fab9f9d8: No bootstrap required, opened a new log
I20260810 18:46:58.720660 2160 ts_tablet_manager.cc:1397] T 304a7a44df824118bf934b6014bb1bf9 P d8bd7f2d079e486dacf92ae5fab9f9d8: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260810 18:46:58.728219 2160 raft_consensus.cc:348] T 304a7a44df824118bf934b6014bb1bf9 P d8bd7f2d079e486dacf92ae5fab9f9d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "957f19c319f04f439dff1c455af3da1a" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42905 } } peers { permanent_uuid: "66720c194ed0469d96e37857ebdbfbea" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 41369 } } peers { permanent_uuid: "d8bd7f2d079e486dacf92ae5fab9f9d8" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 43451 } }
I20260810 18:46:58.728616 2160 raft_consensus.cc:374] T 304a7a44df824118bf934b6014bb1bf9 P d8bd7f2d079e486dacf92ae5fab9f9d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:46:58.728719 2160 raft_consensus.cc:729] T 304a7a44df824118bf934b6014bb1bf9 P d8bd7f2d079e486dacf92ae5fab9f9d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8bd7f2d079e486dacf92ae5fab9f9d8, State: Initialized, Role: FOLLOWER
I20260810 18:46:58.728742 2158 raft_consensus.cc:348] T 304a7a44df824118bf934b6014bb1bf9 P 66720c194ed0469d96e37857ebdbfbea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "957f19c319f04f439dff1c455af3da1a" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42905 } } peers { permanent_uuid: "66720c194ed0469d96e37857ebdbfbea" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 41369 } } peers { permanent_uuid: "d8bd7f2d079e486dacf92ae5fab9f9d8" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 43451 } }
I20260810 18:46:58.729192 2158 raft_consensus.cc:374] T 304a7a44df824118bf934b6014bb1bf9 P 66720c194ed0469d96e37857ebdbfbea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:46:58.729298 2158 raft_consensus.cc:729] T 304a7a44df824118bf934b6014bb1bf9 P 66720c194ed0469d96e37857ebdbfbea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66720c194ed0469d96e37857ebdbfbea, State: Initialized, Role: FOLLOWER
I20260810 18:46:58.729502 2160 consensus_queue.cc:260] T 304a7a44df824118bf934b6014bb1bf9 P d8bd7f2d079e486dacf92ae5fab9f9d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "957f19c319f04f439dff1c455af3da1a" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42905 } } peers { permanent_uuid: "66720c194ed0469d96e37857ebdbfbea" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 41369 } } peers { permanent_uuid: "d8bd7f2d079e486dacf92ae5fab9f9d8" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 43451 } }
I20260810 18:46:58.729921 2158 consensus_queue.cc:260] T 304a7a44df824118bf934b6014bb1bf9 P 66720c194ed0469d96e37857ebdbfbea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "957f19c319f04f439dff1c455af3da1a" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42905 } } peers { permanent_uuid: "66720c194ed0469d96e37857ebdbfbea" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 41369 } } peers { permanent_uuid: "d8bd7f2d079e486dacf92ae5fab9f9d8" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 43451 } }
I20260810 18:46:58.730135 2159 raft_consensus.cc:348] T 304a7a44df824118bf934b6014bb1bf9 P 957f19c319f04f439dff1c455af3da1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "957f19c319f04f439dff1c455af3da1a" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42905 } } peers { permanent_uuid: "66720c194ed0469d96e37857ebdbfbea" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 41369 } } peers { permanent_uuid: "d8bd7f2d079e486dacf92ae5fab9f9d8" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 43451 } }
I20260810 18:46:58.731003 2134 heartbeater.cc:503] Master 127.4.228.254:35143 was elected leader, sending a full tablet report...
I20260810 18:46:58.731428 2160 ts_tablet_manager.cc:1428] T 304a7a44df824118bf934b6014bb1bf9 P d8bd7f2d079e486dacf92ae5fab9f9d8: Time spent starting tablet: real 0.011s user 0.008s sys 0.003s
I20260810 18:46:58.730615 2159 raft_consensus.cc:374] T 304a7a44df824118bf934b6014bb1bf9 P 957f19c319f04f439dff1c455af3da1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:46:58.731802 2159 raft_consensus.cc:729] T 304a7a44df824118bf934b6014bb1bf9 P 957f19c319f04f439dff1c455af3da1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 957f19c319f04f439dff1c455af3da1a, State: Initialized, Role: FOLLOWER
I20260810 18:46:58.732721 2159 consensus_queue.cc:260] T 304a7a44df824118bf934b6014bb1bf9 P 957f19c319f04f439dff1c455af3da1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "957f19c319f04f439dff1c455af3da1a" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42905 } } peers { permanent_uuid: "66720c194ed0469d96e37857ebdbfbea" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 41369 } } peers { permanent_uuid: "d8bd7f2d079e486dacf92ae5fab9f9d8" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 43451 } }
I20260810 18:46:58.735352 1940 heartbeater.cc:503] Master 127.4.228.254:35143 was elected leader, sending a full tablet report...
I20260810 18:46:58.737593 2159 ts_tablet_manager.cc:1428] T 304a7a44df824118bf934b6014bb1bf9 P 957f19c319f04f439dff1c455af3da1a: Time spent starting tablet: real 0.018s user 0.014s sys 0.002s
I20260810 18:46:58.738981 2158 ts_tablet_manager.cc:1428] T 304a7a44df824118bf934b6014bb1bf9 P 66720c194ed0469d96e37857ebdbfbea: Time spent starting tablet: real 0.021s user 0.016s sys 0.004s
I20260810 18:46:58.851613 2168 raft_consensus.cc:482] T 304a7a44df824118bf934b6014bb1bf9 P 66720c194ed0469d96e37857ebdbfbea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:46:58.851963 2168 raft_consensus.cc:504] T 304a7a44df824118bf934b6014bb1bf9 P 66720c194ed0469d96e37857ebdbfbea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "957f19c319f04f439dff1c455af3da1a" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42905 } } peers { permanent_uuid: "66720c194ed0469d96e37857ebdbfbea" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 41369 } } peers { permanent_uuid: "d8bd7f2d079e486dacf92ae5fab9f9d8" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 43451 } }
I20260810 18:46:58.853820 2168 leader_election.cc:288] T 304a7a44df824118bf934b6014bb1bf9 P 66720c194ed0469d96e37857ebdbfbea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 957f19c319f04f439dff1c455af3da1a (127.4.228.193:42905), d8bd7f2d079e486dacf92ae5fab9f9d8 (127.4.228.195:43451)
I20260810 18:46:58.861498 2152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:58.854334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:55375 (local address 127.4.228.193:42905)
0810 18:46:58.854669 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:58.854678 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:58.855180 (+ 502us) server_negotiation.cc:407] Connection header received
0810 18:46:58.855430 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:58.855435 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:58.855542 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:58.855664 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:46:58.856360 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:58.857395 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:58.859550 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:58.860585 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:58.860714 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:58.860766 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:58.861190 (+ 424us) server_negotiation.cc:299] Negotiation successful
0810 18:46:58.861281 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260810 18:46:58.861763 2170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:58.854253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:42905 (local address 127.4.228.194:55375)
0810 18:46:58.855058 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:58.855076 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:58.855269 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:58.855722 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:58.855728 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:58.855762 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:46:58.856199 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:58.856214 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:58.857599 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:58.857606 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:58.859351 (+ 1745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:58.859364 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:58.860807 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:58.860814 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:58.861011 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:58.861053 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 18:46:58.861168 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 18:46:58.861316 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"thread_start_us":487,"threads_started":1}
I20260810 18:46:58.862344 2156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:58.855326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:60865 (local address 127.4.228.195:43451)
0810 18:46:58.855595 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:58.855604 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:58.856013 (+ 409us) server_negotiation.cc:407] Connection header received
0810 18:46:58.856197 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:58.856204 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:58.856302 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:58.856415 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:46:58.857078 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:58.858071 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:58.860218 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:58.861397 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:58.861562 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:58.861900 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:58.862018 (+ 118us) server_negotiation.cc:299] Negotiation successful
0810 18:46:58.862163 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260810 18:46:58.862464 2171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:58.855195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:43451 (local address 127.4.228.194:60865)
0810 18:46:58.855903 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:58.855920 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:46:58.856038 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:46:58.856462 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:46:58.856466 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:46:58.856494 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:46:58.856862 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:58.856875 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:58.858260 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:58.858264 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:58.860012 (+ 1748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:46:58.860027 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:46:58.861611 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:46:58.861620 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:46:58.861818 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:58.861880 (+ 62us) client_negotiation.cc:769] Sending connection context
0810 18:46:58.861973 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 18:46:58.862097 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"mutex_wait_us":1,"thread_start_us":388,"threads_started":1}
I20260810 18:46:58.863096 1673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "304a7a44df824118bf934b6014bb1bf9" candidate_uuid: "66720c194ed0469d96e37857ebdbfbea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "957f19c319f04f439dff1c455af3da1a" is_pre_election: true
I20260810 18:46:58.863430 2065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "304a7a44df824118bf934b6014bb1bf9" candidate_uuid: "66720c194ed0469d96e37857ebdbfbea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8bd7f2d079e486dacf92ae5fab9f9d8" is_pre_election: true
I20260810 18:46:58.863715 1673 raft_consensus.cc:2454] T 304a7a44df824118bf934b6014bb1bf9 P 957f19c319f04f439dff1c455af3da1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66720c194ed0469d96e37857ebdbfbea in term 0.
I20260810 18:46:58.863996 2065 raft_consensus.cc:2454] T 304a7a44df824118bf934b6014bb1bf9 P d8bd7f2d079e486dacf92ae5fab9f9d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66720c194ed0469d96e37857ebdbfbea in term 0.
I20260810 18:46:58.864558 1768 leader_election.cc:302] T 304a7a44df824118bf934b6014bb1bf9 P 66720c194ed0469d96e37857ebdbfbea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66720c194ed0469d96e37857ebdbfbea, 957f19c319f04f439dff1c455af3da1a; no voters:
I20260810 18:46:58.864979 2168 raft_consensus.cc:2781] T 304a7a44df824118bf934b6014bb1bf9 P 66720c194ed0469d96e37857ebdbfbea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:46:58.865100 2168 raft_consensus.cc:482] T 304a7a44df824118bf934b6014bb1bf9 P 66720c194ed0469d96e37857ebdbfbea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:46:58.865262 2168 raft_consensus.cc:3037] T 304a7a44df824118bf934b6014bb1bf9 P 66720c194ed0469d96e37857ebdbfbea [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:46:58.867566 2168 raft_consensus.cc:504] T 304a7a44df824118bf934b6014bb1bf9 P 66720c194ed0469d96e37857ebdbfbea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "957f19c319f04f439dff1c455af3da1a" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42905 } } peers { permanent_uuid: "66720c194ed0469d96e37857ebdbfbea" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 41369 } } peers { permanent_uuid: "d8bd7f2d079e486dacf92ae5fab9f9d8" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 43451 } }
I20260810 18:46:58.868163 2168 leader_election.cc:288] T 304a7a44df824118bf934b6014bb1bf9 P 66720c194ed0469d96e37857ebdbfbea [CANDIDATE]: Term 1 election: Requested vote from peers 957f19c319f04f439dff1c455af3da1a (127.4.228.193:42905), d8bd7f2d079e486dacf92ae5fab9f9d8 (127.4.228.195:43451)
I20260810 18:46:58.868652 1673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "304a7a44df824118bf934b6014bb1bf9" candidate_uuid: "66720c194ed0469d96e37857ebdbfbea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "957f19c319f04f439dff1c455af3da1a"
I20260810 18:46:58.868796 2065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "304a7a44df824118bf934b6014bb1bf9" candidate_uuid: "66720c194ed0469d96e37857ebdbfbea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8bd7f2d079e486dacf92ae5fab9f9d8"
I20260810 18:46:58.868887 1673 raft_consensus.cc:3037] T 304a7a44df824118bf934b6014bb1bf9 P 957f19c319f04f439dff1c455af3da1a [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:46:58.869030 2065 raft_consensus.cc:3037] T 304a7a44df824118bf934b6014bb1bf9 P d8bd7f2d079e486dacf92ae5fab9f9d8 [term 0 FOLLOWER]: Advancing to term 1
W20260810 18:46:58.869053 2136 tablet.cc:2369] T 304a7a44df824118bf934b6014bb1bf9 P d8bd7f2d079e486dacf92ae5fab9f9d8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:46:58.871484 1673 raft_consensus.cc:2454] T 304a7a44df824118bf934b6014bb1bf9 P 957f19c319f04f439dff1c455af3da1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66720c194ed0469d96e37857ebdbfbea in term 1.
I20260810 18:46:58.871615 2065 raft_consensus.cc:2454] T 304a7a44df824118bf934b6014bb1bf9 P d8bd7f2d079e486dacf92ae5fab9f9d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66720c194ed0469d96e37857ebdbfbea in term 1.
I20260810 18:46:58.871963 1768 leader_election.cc:302] T 304a7a44df824118bf934b6014bb1bf9 P 66720c194ed0469d96e37857ebdbfbea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66720c194ed0469d96e37857ebdbfbea, 957f19c319f04f439dff1c455af3da1a; no voters:
I20260810 18:46:58.872303 2168 raft_consensus.cc:2781] T 304a7a44df824118bf934b6014bb1bf9 P 66720c194ed0469d96e37857ebdbfbea [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:46:58.872874 2168 raft_consensus.cc:686] T 304a7a44df824118bf934b6014bb1bf9 P 66720c194ed0469d96e37857ebdbfbea [term 1 LEADER]: Becoming Leader. State: Replica: 66720c194ed0469d96e37857ebdbfbea, State: Running, Role: LEADER
I20260810 18:46:58.873354 2168 consensus_queue.cc:237] T 304a7a44df824118bf934b6014bb1bf9 P 66720c194ed0469d96e37857ebdbfbea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "957f19c319f04f439dff1c455af3da1a" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42905 } } peers { permanent_uuid: "66720c194ed0469d96e37857ebdbfbea" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 41369 } } peers { permanent_uuid: "d8bd7f2d079e486dacf92ae5fab9f9d8" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 43451 } }
I20260810 18:46:58.878685 1458 catalog_manager.cc:5495] T 304a7a44df824118bf934b6014bb1bf9 P 66720c194ed0469d96e37857ebdbfbea reported cstate change: term changed from 0 to 1, leader changed from <none> to 66720c194ed0469d96e37857ebdbfbea (127.4.228.194). New cstate: current_term: 1 leader_uuid: "66720c194ed0469d96e37857ebdbfbea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "957f19c319f04f439dff1c455af3da1a" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66720c194ed0469d96e37857ebdbfbea" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 41369 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8bd7f2d079e486dacf92ae5fab9f9d8" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 43451 } health_report { overall_health: UNKNOWN } } }
W20260810 18:46:58.882820 1942 tablet.cc:2369] T 304a7a44df824118bf934b6014bb1bf9 P 66720c194ed0469d96e37857ebdbfbea: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:46:58.893483 1747 tablet.cc:2369] T 304a7a44df824118bf934b6014bb1bf9 P 957f19c319f04f439dff1c455af3da1a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:46:58.900364 2155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:58.894109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60020 (local address 127.4.228.194:41369)
0810 18:46:58.894341 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:58.894347 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:58.894367 (+ 20us) server_negotiation.cc:407] Connection header received
0810 18:46:58.894682 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:58.894687 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:58.894828 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:58.894981 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:46:58.895413 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:58.896358 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:58.897809 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:58.898237 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:58.898383 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:58.899066 (+ 683us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:46:58.899582 (+ 516us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:46:58.899804 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:58.899974 (+ 170us) server_negotiation.cc:299] Negotiation successful
0810 18:46:58.900022 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260810 18:46:58.909829 2065 raft_consensus.cc:1261] T 304a7a44df824118bf934b6014bb1bf9 P d8bd7f2d079e486dacf92ae5fab9f9d8 [term 1 FOLLOWER]: Refusing update from remote peer 66720c194ed0469d96e37857ebdbfbea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:46:58.909858 1673 raft_consensus.cc:1261] T 304a7a44df824118bf934b6014bb1bf9 P 957f19c319f04f439dff1c455af3da1a [term 1 FOLLOWER]: Refusing update from remote peer 66720c194ed0469d96e37857ebdbfbea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:46:58.910948 2172 consensus_queue.cc:1035] T 304a7a44df824118bf934b6014bb1bf9 P 66720c194ed0469d96e37857ebdbfbea [LEADER]: Connected to new peer: Peer: permanent_uuid: "957f19c319f04f439dff1c455af3da1a" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 42905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:46:58.911312 2168 consensus_queue.cc:1035] T 304a7a44df824118bf934b6014bb1bf9 P 66720c194ed0469d96e37857ebdbfbea [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8bd7f2d079e486dacf92ae5fab9f9d8" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 43451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:46:58.927224 2179 mvcc.cc:205] Tried to move back new op lower bound from 7317043687043588096 to 7317043686909358080. Current Snapshot: MvccSnapshot[applied={T|T < 7317043687043588096}]
I20260810 18:46:58.939253 1457 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33578:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:46:58.939579 1457 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33578:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:46:58.943890 1457 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 134e5cc9d98843ce8911fd144763de39: Sending DeleteTablet for 3 replicas of tablet 304a7a44df824118bf934b6014bb1bf9
I20260810 18:46:58.945320 1842 tablet_service.cc:1514] Processing DeleteTablet for tablet 304a7a44df824118bf934b6014bb1bf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:46:58 UTC) from {username='slave'} at 127.0.0.1:60008
I20260810 18:46:58.945442 2039 tablet_service.cc:1514] Processing DeleteTablet for tablet 304a7a44df824118bf934b6014bb1bf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:46:58 UTC) from {username='slave'} at 127.0.0.1:51144
I20260810 18:46:58.945442 1644 tablet_service.cc:1514] Processing DeleteTablet for tablet 304a7a44df824118bf934b6014bb1bf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:46:58 UTC) from {username='slave'} at 127.0.0.1:60076
I20260810 18:46:58.948688 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 1550
I20260810 18:46:58.959592 2190 tablet_replica.cc:330] T 304a7a44df824118bf934b6014bb1bf9 P 66720c194ed0469d96e37857ebdbfbea: stopping tablet replica
I20260810 18:46:58.960130 2190 raft_consensus.cc:2229] T 304a7a44df824118bf934b6014bb1bf9 P 66720c194ed0469d96e37857ebdbfbea [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:46:58.961002 2190 raft_consensus.cc:2258] T 304a7a44df824118bf934b6014bb1bf9 P 66720c194ed0469d96e37857ebdbfbea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:46:58.963572 2190 ts_tablet_manager.cc:1905] T 304a7a44df824118bf934b6014bb1bf9 P 66720c194ed0469d96e37857ebdbfbea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:46:58.971391 2190 ts_tablet_manager.cc:1918] T 304a7a44df824118bf934b6014bb1bf9 P 66720c194ed0469d96e37857ebdbfbea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:46:58.971586 2190 log.cc:1196] T 304a7a44df824118bf934b6014bb1bf9 P 66720c194ed0469d96e37857ebdbfbea: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-1/wal/wals/304a7a44df824118bf934b6014bb1bf9
I20260810 18:46:58.972195 2190 ts_tablet_manager.cc:1939] T 304a7a44df824118bf934b6014bb1bf9 P 66720c194ed0469d96e37857ebdbfbea: Deleting consensus metadata
I20260810 18:46:58.972440 2194 tablet_replica.cc:330] T 304a7a44df824118bf934b6014bb1bf9 P 957f19c319f04f439dff1c455af3da1a: stopping tablet replica
I20260810 18:46:58.972831 2193 tablet_replica.cc:330] T 304a7a44df824118bf934b6014bb1bf9 P d8bd7f2d079e486dacf92ae5fab9f9d8: stopping tablet replica
I20260810 18:46:58.972954 2194 raft_consensus.cc:2229] T 304a7a44df824118bf934b6014bb1bf9 P 957f19c319f04f439dff1c455af3da1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:46:58.973369 2193 raft_consensus.cc:2229] T 304a7a44df824118bf934b6014bb1bf9 P d8bd7f2d079e486dacf92ae5fab9f9d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:46:58.973537 2194 raft_consensus.cc:2258] T 304a7a44df824118bf934b6014bb1bf9 P 957f19c319f04f439dff1c455af3da1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:46:58.974087 2193 raft_consensus.cc:2258] T 304a7a44df824118bf934b6014bb1bf9 P d8bd7f2d079e486dacf92ae5fab9f9d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:46:58.974143 1436 catalog_manager.cc:4841] TS 66720c194ed0469d96e37857ebdbfbea (127.4.228.194:41369): tablet 304a7a44df824118bf934b6014bb1bf9 (table test-table [id=354b70701fc7494bbd8315f95d88738e]) successfully deleted
I20260810 18:46:58.976873 2194 ts_tablet_manager.cc:1905] T 304a7a44df824118bf934b6014bb1bf9 P 957f19c319f04f439dff1c455af3da1a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:46:58.976876 2193 ts_tablet_manager.cc:1905] T 304a7a44df824118bf934b6014bb1bf9 P d8bd7f2d079e486dacf92ae5fab9f9d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:46:58.981715 2152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:58.950781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60088 (local address 127.4.228.193:42905)
0810 18:46:58.951029 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:58.951038 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:58.970133 (+ 19095us) server_negotiation.cc:407] Connection header received
0810 18:46:58.970824 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:58.970834 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:58.971020 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:58.971120 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:58.973968 (+ 2848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:58.975508 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:58.977478 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:58.978159 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:58.978285 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:58.979884 (+ 1599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:58.979898 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:58.979917 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:46:58.980703 (+ 786us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:58.980770 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:46:58.980803 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:58.980897 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:58.981169 (+ 272us) server_negotiation.cc:299] Negotiation successful
0810 18:46:58.981313 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 18:46:58.983074 1736 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:46:58.987573 2193 ts_tablet_manager.cc:1918] T 304a7a44df824118bf934b6014bb1bf9 P d8bd7f2d079e486dacf92ae5fab9f9d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:46:58.987763 2193 log.cc:1196] T 304a7a44df824118bf934b6014bb1bf9 P d8bd7f2d079e486dacf92ae5fab9f9d8: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-2/wal/wals/304a7a44df824118bf934b6014bb1bf9
I20260810 18:46:58.988298 2193 ts_tablet_manager.cc:1939] T 304a7a44df824118bf934b6014bb1bf9 P d8bd7f2d079e486dacf92ae5fab9f9d8: Deleting consensus metadata
I20260810 18:46:58.990317 1436 catalog_manager.cc:4841] TS d8bd7f2d079e486dacf92ae5fab9f9d8 (127.4.228.195:43451): tablet 304a7a44df824118bf934b6014bb1bf9 (table test-table [id=354b70701fc7494bbd8315f95d88738e]) successfully deleted
I20260810 18:46:59.074636 2194 ts_tablet_manager.cc:1918] T 304a7a44df824118bf934b6014bb1bf9 P 957f19c319f04f439dff1c455af3da1a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:46:59.074934 2194 log.cc:1196] T 304a7a44df824118bf934b6014bb1bf9 P 957f19c319f04f439dff1c455af3da1a: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387267874959-5011-0/minicluster-data/ts-0/wal/wals/304a7a44df824118bf934b6014bb1bf9
I20260810 18:46:59.075416 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 1550
I20260810 18:46:59.075630 2194 ts_tablet_manager.cc:1939] T 304a7a44df824118bf934b6014bb1bf9 P 957f19c319f04f439dff1c455af3da1a: Deleting consensus metadata
W20260810 18:46:59.110010 1436 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.228.193:42905 (error 108)
I20260810 18:46:59.110642 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 1751
I20260810 18:46:59.112458 2157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:59.111852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:42905 (local address 127.0.0.1:60100)
0810 18:46:59.112124 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:59.112262 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.193:42905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
W20260810 18:46:59.112756 1436 catalog_manager.cc:4584] TS 957f19c319f04f439dff1c455af3da1a (127.4.228.193:42905): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 304a7a44df824118bf934b6014bb1bf9: Network error: Client connection negotiation failed: client connection to 127.4.228.193:42905: connect: Connection refused (error 111)
I20260810 18:46:59.120056 2155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:59.111889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60028 (local address 127.4.228.194:41369)
0810 18:46:59.112114 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:59.112120 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:59.112143 (+ 23us) server_negotiation.cc:407] Connection header received
0810 18:46:59.112394 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:59.112406 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:59.112559 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:59.112681 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:59.113352 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:59.114682 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:59.116105 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:59.116775 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:59.116910 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:59.118188 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:59.118201 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:59.118222 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:46:59.119109 (+ 887us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:59.119186 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:46:59.119231 (+ 45us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:59.119349 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:59.119544 (+ 195us) server_negotiation.cc:299] Negotiation successful
0810 18:46:59.119652 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260810 18:46:59.120906 1930 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:46:59.166530 2157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:59.165912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:42905 (local address 127.0.0.1:60102)
0810 18:46:59.166169 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:59.166287 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.193:42905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260810 18:46:59.167846 2157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:59.167409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:42905 (local address 127.0.0.1:60114)
0810 18:46:59.167587 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:59.167683 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.193:42905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260810 18:46:59.228832 2157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:59.228198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:42905 (local address 127.0.0.1:60118)
0810 18:46:59.228477 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:59.228606 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.193:42905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260810 18:46:59.230074 2157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:59.229609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:42905 (local address 127.0.0.1:60128)
0810 18:46:59.229804 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:59.229908 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.193:42905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260810 18:46:59.238839 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 1751
I20260810 18:46:59.266994 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 1944
I20260810 18:46:59.274273 2156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:59.267919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51146 (local address 127.4.228.195:43451)
0810 18:46:59.268110 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 18:46:59.268115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:46:59.268135 (+ 20us) server_negotiation.cc:407] Connection header received
0810 18:46:59.268337 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:46:59.268342 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:46:59.268432 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:46:59.268519 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:46:59.268981 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:59.269727 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:59.271183 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:46:59.271572 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:46:59.271714 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:46:59.272809 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:46:59.272819 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:46:59.272835 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:46:59.273444 (+ 609us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:46:59.273492 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:46:59.273519 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:46:59.273607 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 18:46:59.273797 (+ 190us) server_negotiation.cc:299] Negotiation successful
0810 18:46:59.273891 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 18:46:59.275076 2129 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:46:59.311295 2157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:59.310681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:42905 (local address 127.0.0.1:60140)
0810 18:46:59.310910 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:59.311109 (+ 199us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.193:42905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260810 18:46:59.312436 2157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:59.312020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:42905 (local address 127.0.0.1:60150)
0810 18:46:59.312249 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:59.312321 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.193:42905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260810 18:46:59.373417 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 1944
I20260810 18:46:59.401481 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 1412
I20260810 18:46:59.402849 1500 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:46:59.517295 2157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:46:59.516538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:42905 (local address 127.0.0.1:60164)
0810 18:46:59.516895 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0810 18:46:59.517027 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.193:42905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187}
I20260810 18:46:59.517344 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 1412
2026-08-10T18:46:59Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3037 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:46:59 dist-test-slave-7kzw krb5kdc[2203](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:46:59 dist-test-slave-7kzw krb5kdc[2203](info): set up 2 sockets
Aug 10 18:46:59 dist-test-slave-7kzw krb5kdc[2203](info): commencing operation
krb5kdc: starting...
W20260810 18:47:00.899096 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.328s user 0.003s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:47:01 dist-test-slave-7kzw krb5kdc[2203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:47:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:47:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.228.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
I20260810 18:47:01.233594 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:40343
--webserver_interface=127.4.228.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:34691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:40343
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:47:01.591856 2226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:47:01.592213 2226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:47:01.592370 2226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:47:01.599632 2226 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:47:01.599781 2226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:47:01.599880 2226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:47:01.599980 2226 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:47:01.600057 2226 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:47:01.610114 2226 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:34691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:40343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:40343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:47:01.611548 2226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:47:01.612941 2226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:47:01.625360 2232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:01.625490 2233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:01.625502 2237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:47:01.626602 2226 server_base.cc:1034] running on GCE node
I20260810 18:47:01.627420 2226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:47:01.629015 2226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:47:01.630266 2226 hybrid_clock.cc:648] HybridClock initialized: now 1786387621630220 us; error 42 us; skew 500 ppm
I20260810 18:47:01.630700 2226 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:47:01 dist-test-slave-7kzw krb5kdc[2203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387621, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:47:01.633986 2226 init.cc:377] Logged in from keytab as kudu/127.4.228.254@KRBTEST.COM (short username kudu)
I20260810 18:47:01.635624 2226 webserver.cc:460] Webserver started at http://127.4.228.254:42709/ using document root <none> and password file <none>
I20260810 18:47:01.636365 2226 fs_manager.cc:359] Metadata directory not provided
I20260810 18:47:01.636530 2226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:47:01.636864 2226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:47:01.640148 2226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "6579ad59ed4a44449947bce31c1f463f"
format_stamp: "Formatted at 2026-08-10 18:47:01 on dist-test-slave-7kzw"
server_key: "cee6904d28a28db1a585e25865b4e86c"
server_key_iv: "a7e9765f0485ed5811d4b796fd54f33a"
server_key_version: "encryptionkey@0"
I20260810 18:47:01.641083 2226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "6579ad59ed4a44449947bce31c1f463f"
format_stamp: "Formatted at 2026-08-10 18:47:01 on dist-test-slave-7kzw"
server_key: "cee6904d28a28db1a585e25865b4e86c"
server_key_iv: "a7e9765f0485ed5811d4b796fd54f33a"
server_key_version: "encryptionkey@0"
I20260810 18:47:01.646560 2226 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 18:47:01.650147 2246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:01.651654 2226 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:47:01.651887 2226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "6579ad59ed4a44449947bce31c1f463f"
format_stamp: "Formatted at 2026-08-10 18:47:01 on dist-test-slave-7kzw"
server_key: "cee6904d28a28db1a585e25865b4e86c"
server_key_iv: "a7e9765f0485ed5811d4b796fd54f33a"
server_key_version: "encryptionkey@0"
I20260810 18:47:01.652094 2226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:47:01.680516 2226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:47:01.684953 2226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:47:01.685272 2226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:47:01.704921 2226 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:40343
I20260810 18:47:01.704952 2318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:40343 every 8 connection(s)
I20260810 18:47:01.706357 2226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:47:01.710875 2319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:47:01.716463 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 2226
I20260810 18:47:01.716882 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:47:01.717504 5011 external_mini_cluster.cc:1428] Setting key e4ccba670288a79b8fafc8724f9ec246
I20260810 18:47:01.723326 2319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6579ad59ed4a44449947bce31c1f463f: Bootstrap starting.
I20260810 18:47:01.727240 2319 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6579ad59ed4a44449947bce31c1f463f: Neither blocks nor log segments found. Creating new log.
I20260810 18:47:01.728570 2319 log.cc:824] T 00000000000000000000000000000000 P 6579ad59ed4a44449947bce31c1f463f: Log is configured to *not* fsync() on all Append() calls
I20260810 18:47:01.732127 2319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6579ad59ed4a44449947bce31c1f463f: No bootstrap required, opened a new log
Aug 10 18:47:01 dist-test-slave-7kzw krb5kdc[2203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:47:01.740736 2319 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6579ad59ed4a44449947bce31c1f463f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6579ad59ed4a44449947bce31c1f463f" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 40343 } }
I20260810 18:47:01.741123 2319 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6579ad59ed4a44449947bce31c1f463f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:47:01.741258 2319 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6579ad59ed4a44449947bce31c1f463f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6579ad59ed4a44449947bce31c1f463f, State: Initialized, Role: FOLLOWER
I20260810 18:47:01.741902 2319 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6579ad59ed4a44449947bce31c1f463f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6579ad59ed4a44449947bce31c1f463f" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 40343 } }
I20260810 18:47:01.742167 2319 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6579ad59ed4a44449947bce31c1f463f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:47:01.742344 2319 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6579ad59ed4a44449947bce31c1f463f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:47:01.742627 2319 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6579ad59ed4a44449947bce31c1f463f [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:47:01.744500 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:01.719759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36044 (local address 127.4.228.254:40343)
0810 18:47:01.720776 (+ 1017us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:01.720788 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:01.720839 (+ 51us) server_negotiation.cc:407] Connection header received
0810 18:47:01.722839 (+ 2000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:01.722865 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:01.723281 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:01.723694 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:01.725639 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:01.727239 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:01.729204 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:01.730417 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:01.730729 (+ 312us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:01.735524 (+ 4795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:01.735532 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:01.735547 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:47:01.735590 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:01.739468 (+ 3878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:01.740194 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:01.740201 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:01.740208 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:01.740439 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:01.740896 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:01.740902 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:01.740905 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:01.741684 (+ 779us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:01.741997 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:01.742563 (+ 566us) server_negotiation.cc:299] Negotiation successful
0810 18:47:01.743057 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":746,"thread_start_us":496,"threads_started":1}
I20260810 18:47:01.745687 2319 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6579ad59ed4a44449947bce31c1f463f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6579ad59ed4a44449947bce31c1f463f" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 40343 } }
I20260810 18:47:01.746253 2319 leader_election.cc:302] T 00000000000000000000000000000000 P 6579ad59ed4a44449947bce31c1f463f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6579ad59ed4a44449947bce31c1f463f; no voters:
I20260810 18:47:01.746905 2319 leader_election.cc:288] T 00000000000000000000000000000000 P 6579ad59ed4a44449947bce31c1f463f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:47:01.747063 2327 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6579ad59ed4a44449947bce31c1f463f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:47:01.748142 2327 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6579ad59ed4a44449947bce31c1f463f [term 1 LEADER]: Becoming Leader. State: Replica: 6579ad59ed4a44449947bce31c1f463f, State: Running, Role: LEADER
I20260810 18:47:01.748708 2327 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6579ad59ed4a44449947bce31c1f463f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6579ad59ed4a44449947bce31c1f463f" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 40343 } }
I20260810 18:47:01.749153 2319 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6579ad59ed4a44449947bce31c1f463f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:47:01.751374 2328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6579ad59ed4a44449947bce31c1f463f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6579ad59ed4a44449947bce31c1f463f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6579ad59ed4a44449947bce31c1f463f" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 40343 } } }
I20260810 18:47:01.751749 2328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6579ad59ed4a44449947bce31c1f463f [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:01.752249 2329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6579ad59ed4a44449947bce31c1f463f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6579ad59ed4a44449947bce31c1f463f. Latest consensus state: current_term: 1 leader_uuid: "6579ad59ed4a44449947bce31c1f463f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6579ad59ed4a44449947bce31c1f463f" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 40343 } } }
I20260810 18:47:01.752588 2329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6579ad59ed4a44449947bce31c1f463f [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:01.765512 2334 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:47:01.771942 2334 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:47:01.781859 2334 catalog_manager.cc:1347] Generated new cluster ID: c4a4ee68ed3344089a507b650d3d37c7
I20260810 18:47:01.781991 2334 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:47:01.809788 2334 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:47:01.811272 2334 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:47:01.821132 2334 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6579ad59ed4a44449947bce31c1f463f: Generated new TSK 0
I20260810 18:47:01.821950 2334 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.228.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab.
Entry for principal kudu/127.4.228.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab.
I20260810 18:47:01.917544 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.193:0
--local_ip_for_outbound_sockets=127.4.228.193
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:40343
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:34691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:47:02.283398 2364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:47:02.283703 2364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:47:02.283859 2364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:47:02.291196 2364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:47:02.291301 2364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:47:02.291473 2364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.193
I20260810 18:47:02.301170 2364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:34691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.193.keytab
--principal=kudu/127.4.228.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:40343
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.193
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:47:02.302552 2364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:47:02.303960 2364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:47:02.317663 2373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:02.317663 2374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:02.317725 2376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:47:02.318588 2364 server_base.cc:1034] running on GCE node
I20260810 18:47:02.319247 2364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:47:02.320401 2364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:47:02.321653 2364 hybrid_clock.cc:648] HybridClock initialized: now 1786387622321561 us; error 82 us; skew 500 ppm
I20260810 18:47:02.322045 2364 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:47:02 dist-test-slave-7kzw krb5kdc[2203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387622, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:47:02.325419 2364 init.cc:377] Logged in from keytab as kudu/127.4.228.193@KRBTEST.COM (short username kudu)
I20260810 18:47:02.327036 2364 webserver.cc:460] Webserver started at http://127.4.228.193:43275/ using document root <none> and password file <none>
I20260810 18:47:02.327767 2364 fs_manager.cc:359] Metadata directory not provided
I20260810 18:47:02.327920 2364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:47:02.328267 2364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:47:02.331279 2364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-0/data/instance:
uuid: "c2dbf46fbbb64587b0d75e13af7fcc9f"
format_stamp: "Formatted at 2026-08-10 18:47:02 on dist-test-slave-7kzw"
server_key: "244b683689cc142ac6e5289384d3d2d7"
server_key_iv: "4fbc237278d8c225162227d75a4ba96c"
server_key_version: "encryptionkey@0"
I20260810 18:47:02.332123 2364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance:
uuid: "c2dbf46fbbb64587b0d75e13af7fcc9f"
format_stamp: "Formatted at 2026-08-10 18:47:02 on dist-test-slave-7kzw"
server_key: "244b683689cc142ac6e5289384d3d2d7"
server_key_iv: "4fbc237278d8c225162227d75a4ba96c"
server_key_version: "encryptionkey@0"
I20260810 18:47:02.337514 2364 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 18:47:02.341150 2386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:02.342734 2364 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:47:02.342908 2364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-0/wal
uuid: "c2dbf46fbbb64587b0d75e13af7fcc9f"
format_stamp: "Formatted at 2026-08-10 18:47:02 on dist-test-slave-7kzw"
server_key: "244b683689cc142ac6e5289384d3d2d7"
server_key_iv: "4fbc237278d8c225162227d75a4ba96c"
server_key_version: "encryptionkey@0"
I20260810 18:47:02.343086 2364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:47:02.355108 2364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:47:02.360385 2364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:47:02.360718 2364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:47:02.361786 2364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:47:02.363263 2364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:47:02.363353 2364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:02.363440 2364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:47:02.363489 2364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:02.399664 2364 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.193:36491
I20260810 18:47:02.399763 2553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.193:36491 every 8 connection(s)
I20260810 18:47:02.401083 2364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
I20260810 18:47:02.403030 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 2364
I20260810 18:47:02.403234 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance
I20260810 18:47:02.403748 5011 external_mini_cluster.cc:1428] Setting key 0e61421ca3e63e00eccf02b9aef9f8fd
Aug 10 18:47:02 dist-test-slave-7kzw krb5kdc[2203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387622, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.193@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:47:02.424053 2559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:02.404220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:37669 (local address 127.4.228.254:40343)
0810 18:47:02.404806 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:02.404811 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:02.405944 (+ 1133us) server_negotiation.cc:407] Connection header received
0810 18:47:02.408011 (+ 2067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:02.408017 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:02.408118 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:02.408240 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:02.409796 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:02.410616 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:02.412219 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:02.412652 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:02.412802 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:02.416997 (+ 4195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:02.417002 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:02.417006 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:47:02.417043 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:02.420577 (+ 3534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:02.421594 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:02.421598 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:02.421600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:02.421727 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:02.422253 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:02.422257 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:02.422259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:02.422589 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:02.422708 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:02.423617 (+ 909us) server_negotiation.cc:299] Negotiation successful
0810 18:47:02.423789 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":368,"threads_started":1}
I20260810 18:47:02.425788 2560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:02.404868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:40343 (local address 127.4.228.193:37669)
0810 18:47:02.405754 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:02.405813 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 18:47:02.407735 (+ 1922us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:47:02.408485 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:47:02.408494 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:47:02.409019 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:47:02.409618 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:02.409636 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:02.410781 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:02.410791 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:02.412043 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:02.412056 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:02.412906 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:02.412916 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:02.413170 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:02.413956 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:47:02.413984 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:47:02.416741 (+ 2757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:47:02.420776 (+ 4035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:47:02.420789 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:47:02.420810 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:47:02.421474 (+ 664us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:47:02.421850 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:47:02.421858 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:47:02.421865 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:47:02.422123 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:47:02.422725 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:47:02.422736 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:47:02.423214 (+ 478us) client_negotiation.cc:769] Sending connection context
0810 18:47:02.423531 (+ 317us) client_negotiation.cc:240] Negotiation successful
0810 18:47:02.424188 (+ 657us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":323,"threads_started":1}
I20260810 18:47:02.427618 2554 heartbeater.cc:347] Connected to a master server at 127.4.228.254:40343
I20260810 18:47:02.427994 2554 heartbeater.cc:464] Registering TS with master...
I20260810 18:47:02.428767 2554 heartbeater.cc:511] Master 127.4.228.254:40343 requested a full tablet report, sending...
I20260810 18:47:02.431389 2270 ts_manager.cc:194] Registered new tserver with Master: c2dbf46fbbb64587b0d75e13af7fcc9f (127.4.228.193:36491)
I20260810 18:47:02.433621 2270 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.193@KRBTEST.COM'} at 127.4.228.193:37669
WARNING: no policy specified for kudu/127.4.228.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab.
Entry for principal kudu/127.4.228.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab.
I20260810 18:47:02.481200 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.194:0
--local_ip_for_outbound_sockets=127.4.228.194
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:40343
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:34691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:47:02.841338 2567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:47:02.841630 2567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:47:02.841806 2567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:47:02.849359 2567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:47:02.849470 2567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:47:02.849678 2567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.194
I20260810 18:47:02.859597 2567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:34691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.194.keytab
--principal=kudu/127.4.228.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:40343
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.194
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:47:02.860924 2567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:47:02.862293 2567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:47:02.877396 2573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:02.877452 2575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:02.877485 2572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:47:02.878536 2567 server_base.cc:1034] running on GCE node
I20260810 18:47:02.879307 2567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:47:02.880386 2567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:47:02.881680 2567 hybrid_clock.cc:648] HybridClock initialized: now 1786387622881623 us; error 77 us; skew 500 ppm
I20260810 18:47:02.882053 2567 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:47:02 dist-test-slave-7kzw krb5kdc[2203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387622, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:47:02.885602 2567 init.cc:377] Logged in from keytab as kudu/127.4.228.194@KRBTEST.COM (short username kudu)
I20260810 18:47:02.887344 2567 webserver.cc:460] Webserver started at http://127.4.228.194:43191/ using document root <none> and password file <none>
I20260810 18:47:02.888106 2567 fs_manager.cc:359] Metadata directory not provided
I20260810 18:47:02.888262 2567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:47:02.888619 2567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:47:02.891712 2567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-1/data/instance:
uuid: "cfda254e88174172a892a116458dcf2e"
format_stamp: "Formatted at 2026-08-10 18:47:02 on dist-test-slave-7kzw"
server_key: "d40a52fa4514ca97d824822267918198"
server_key_iv: "218e9a1e0e890f6872025a9c61c14925"
server_key_version: "encryptionkey@0"
I20260810 18:47:02.892514 2567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance:
uuid: "cfda254e88174172a892a116458dcf2e"
format_stamp: "Formatted at 2026-08-10 18:47:02 on dist-test-slave-7kzw"
server_key: "d40a52fa4514ca97d824822267918198"
server_key_iv: "218e9a1e0e890f6872025a9c61c14925"
server_key_version: "encryptionkey@0"
I20260810 18:47:02.897854 2567 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 18:47:02.901499 2584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:02.903100 2567 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:47:02.903314 2567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-1/wal
uuid: "cfda254e88174172a892a116458dcf2e"
format_stamp: "Formatted at 2026-08-10 18:47:02 on dist-test-slave-7kzw"
server_key: "d40a52fa4514ca97d824822267918198"
server_key_iv: "218e9a1e0e890f6872025a9c61c14925"
server_key_version: "encryptionkey@0"
I20260810 18:47:02.903534 2567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:47:02.920436 2567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:47:02.924787 2567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:47:02.925084 2567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:47:02.926096 2567 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:47:02.927591 2567 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:47:02.927702 2567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:02.927861 2567 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:47:02.927917 2567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:02.963199 2567 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.194:38595
I20260810 18:47:02.963325 2743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.194:38595 every 8 connection(s)
I20260810 18:47:02.964558 2567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
I20260810 18:47:02.973819 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 2567
I20260810 18:47:02.974010 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance
I20260810 18:47:02.974684 5011 external_mini_cluster.cc:1428] Setting key fe2078d06f3ee0bdf20ea8084dbbabb2
Aug 10 18:47:02 dist-test-slave-7kzw krb5kdc[2203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387622, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.194@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:47:02.987088 2746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:02.967427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:43143 (local address 127.4.228.254:40343)
0810 18:47:02.967877 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:02.967882 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:02.968741 (+ 859us) server_negotiation.cc:407] Connection header received
0810 18:47:02.970731 (+ 1990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:02.970737 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:02.970831 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:02.970955 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:02.972456 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:02.973196 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:02.975577 (+ 2381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:02.976175 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:02.976290 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:02.980476 (+ 4186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:02.980481 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:02.980485 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:47:02.980520 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:02.983365 (+ 2845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:02.984423 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:02.984428 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:02.984430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:02.984572 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:02.985137 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:02.985142 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:02.985144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:02.985461 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:02.985581 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:02.986619 (+ 1038us) server_negotiation.cc:299] Negotiation successful
0810 18:47:02.986810 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":243,"threads_started":1}
I20260810 18:47:02.988623 2747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:02.967833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:40343 (local address 127.4.228.194:43143)
0810 18:47:02.968539 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:02.968586 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 18:47:02.970394 (+ 1808us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:47:02.971195 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:47:02.971203 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:47:02.971729 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:47:02.972276 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:02.972293 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:02.973413 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:02.973419 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:02.975423 (+ 2004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:02.975436 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:02.976398 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:02.976403 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:02.976716 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:02.977435 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:47:02.977457 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:47:02.980221 (+ 2764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:47:02.983527 (+ 3306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:47:02.983539 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:47:02.983560 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:47:02.984258 (+ 698us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:47:02.984715 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:47:02.984722 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:47:02.984729 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:47:02.985005 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:47:02.985603 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:47:02.985614 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:47:02.986101 (+ 487us) client_negotiation.cc:769] Sending connection context
0810 18:47:02.986439 (+ 338us) client_negotiation.cc:240] Negotiation successful
0810 18:47:02.987066 (+ 627us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":278,"threads_started":1}
I20260810 18:47:02.990319 2744 heartbeater.cc:347] Connected to a master server at 127.4.228.254:40343
I20260810 18:47:02.990734 2744 heartbeater.cc:464] Registering TS with master...
I20260810 18:47:02.991549 2744 heartbeater.cc:511] Master 127.4.228.254:40343 requested a full tablet report, sending...
I20260810 18:47:02.993300 2270 ts_manager.cc:194] Registered new tserver with Master: cfda254e88174172a892a116458dcf2e (127.4.228.194:38595)
I20260810 18:47:02.994163 2270 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.194@KRBTEST.COM'} at 127.4.228.194:43143
WARNING: no policy specified for kudu/127.4.228.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab.
Entry for principal kudu/127.4.228.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab.
I20260810 18:47:03.049718 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.195:0
--local_ip_for_outbound_sockets=127.4.228.195
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:40343
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:34691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:47:03.401980 2750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:47:03.402360 2750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:47:03.402554 2750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:47:03.405380 2549 messenger.cc:188] Invalid argument: socket address is ambiguous: no data on pending connections for acceptor pool at 127.4.228.193:36491
W20260810 18:47:03.412087 2750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:47:03.412192 2750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:47:03.412304 2750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.195
I20260810 18:47:03.422057 2750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:34691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.195.keytab
--principal=kudu/127.4.228.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:40343
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.195
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:47:03.423424 2750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:47:03.424731 2750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 18:47:03.436906 2554 heartbeater.cc:503] Master 127.4.228.254:40343 was elected leader, sending a full tablet report...
W20260810 18:47:03.438922 2762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:03.439611 2765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:03.439883 2761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:47:03.441419 2750 server_base.cc:1034] running on GCE node
I20260810 18:47:03.442072 2750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:47:03.443156 2750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:47:03.444402 2750 hybrid_clock.cc:648] HybridClock initialized: now 1786387623444347 us; error 47 us; skew 500 ppm
I20260810 18:47:03.444716 2750 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:47:03 dist-test-slave-7kzw krb5kdc[2203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387623, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:47:03.448256 2750 init.cc:377] Logged in from keytab as kudu/127.4.228.195@KRBTEST.COM (short username kudu)
I20260810 18:47:03.450062 2750 webserver.cc:460] Webserver started at http://127.4.228.195:40101/ using document root <none> and password file <none>
I20260810 18:47:03.450928 2750 fs_manager.cc:359] Metadata directory not provided
I20260810 18:47:03.451040 2750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:47:03.451471 2750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:47:03.454509 2750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-2/data/instance:
uuid: "4c1763425d9b4cb08fabbf6d8bbe8d5a"
format_stamp: "Formatted at 2026-08-10 18:47:03 on dist-test-slave-7kzw"
server_key: "18c174ea6f8eb327f82c1cbad6e0db18"
server_key_iv: "5042cca1264c16e66d1d6fbbaaeab40e"
server_key_version: "encryptionkey@0"
I20260810 18:47:03.455416 2750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance:
uuid: "4c1763425d9b4cb08fabbf6d8bbe8d5a"
format_stamp: "Formatted at 2026-08-10 18:47:03 on dist-test-slave-7kzw"
server_key: "18c174ea6f8eb327f82c1cbad6e0db18"
server_key_iv: "5042cca1264c16e66d1d6fbbaaeab40e"
server_key_version: "encryptionkey@0"
I20260810 18:47:03.460790 2750 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 18:47:03.464490 2774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:03.466007 2750 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:47:03.466233 2750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-2/wal
uuid: "4c1763425d9b4cb08fabbf6d8bbe8d5a"
format_stamp: "Formatted at 2026-08-10 18:47:03 on dist-test-slave-7kzw"
server_key: "18c174ea6f8eb327f82c1cbad6e0db18"
server_key_iv: "5042cca1264c16e66d1d6fbbaaeab40e"
server_key_version: "encryptionkey@0"
I20260810 18:47:03.466405 2750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:47:03.491858 2750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:47:03.497941 2750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:47:03.498325 2750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:47:03.499782 2750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:47:03.501883 2750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:47:03.501991 2750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:03.502136 2750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:47:03.502214 2750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:03.543573 2750 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.195:34545
I20260810 18:47:03.543680 2940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.195:34545 every 8 connection(s)
I20260810 18:47:03.545399 2750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
I20260810 18:47:03.553362 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 2750
I20260810 18:47:03.553561 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance
I20260810 18:47:03.554035 5011 external_mini_cluster.cc:1428] Setting key 32eb5ec045a4990dd2063690fccaf132
Aug 10 18:47:03 dist-test-slave-7kzw krb5kdc[2203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387623, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.195@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:47:03.571233 2945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:03.548909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:51139 (local address 127.4.228.254:40343)
0810 18:47:03.549469 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:03.549474 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:03.550411 (+ 937us) server_negotiation.cc:407] Connection header received
0810 18:47:03.552338 (+ 1927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:03.552343 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:03.552455 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:03.552581 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:03.554308 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:03.555383 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:03.557470 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:03.557983 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:03.558097 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:03.564808 (+ 6711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:03.564814 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:03.564817 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:47:03.564853 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:03.567673 (+ 2820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:03.568553 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:03.568560 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:03.568564 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:03.568704 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:03.569169 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:03.569175 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:03.569178 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:03.569595 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:03.569781 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:03.570616 (+ 835us) server_negotiation.cc:299] Negotiation successful
0810 18:47:03.570846 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":308,"threads_started":1}
I20260810 18:47:03.572296 2946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:03.549421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:40343 (local address 127.4.228.195:51139)
0810 18:47:03.550242 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:03.550280 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 18:47:03.552057 (+ 1777us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:47:03.552829 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:47:03.552838 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:47:03.553377 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:47:03.554053 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:03.554075 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:03.555572 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:03.555576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:03.557304 (+ 1728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:03.557341 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:03.558235 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:03.558240 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:03.558440 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:03.562224 (+ 3784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:47:03.562250 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:47:03.564581 (+ 2331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:47:03.567827 (+ 3246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:47:03.567838 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:47:03.567855 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:47:03.568418 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:47:03.568824 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:47:03.568829 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:47:03.568833 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:47:03.569041 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:47:03.569756 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:47:03.569802 (+ 46us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:47:03.570178 (+ 376us) client_negotiation.cc:769] Sending connection context
0810 18:47:03.570473 (+ 295us) client_negotiation.cc:240] Negotiation successful
0810 18:47:03.571161 (+ 688us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":333,"threads_started":1}
I20260810 18:47:03.573817 2942 heartbeater.cc:347] Connected to a master server at 127.4.228.254:40343
I20260810 18:47:03.574123 2942 heartbeater.cc:464] Registering TS with master...
I20260810 18:47:03.574889 2942 heartbeater.cc:511] Master 127.4.228.254:40343 requested a full tablet report, sending...
I20260810 18:47:03.576376 2270 ts_manager.cc:194] Registered new tserver with Master: 4c1763425d9b4cb08fabbf6d8bbe8d5a (127.4.228.195:34545)
I20260810 18:47:03.577188 2270 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.228.195@KRBTEST.COM'} at 127.4.228.195:51139
I20260810 18:47:03.582432 5011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:47:03.601706 2945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:03.587491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52684 (local address 127.4.228.254:40343)
0810 18:47:03.587713 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:03.587723 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:03.588158 (+ 435us) server_negotiation.cc:407] Connection header received
0810 18:47:03.588374 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:03.588380 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:03.588462 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:03.588572 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:03.590175 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:03.590985 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:03.592299 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:03.592759 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:03.592874 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:03.595592 (+ 2718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:03.595597 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:03.595601 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:47:03.595637 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:03.598665 (+ 3028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:03.599485 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:03.599490 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:03.599492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:03.599643 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:03.600070 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:03.600075 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:03.600077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:03.600444 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:03.600676 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:03.601157 (+ 481us) server_negotiation.cc:299] Negotiation successful
0810 18:47:03.601372 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 18:47:03.606916 2270 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52684:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:47:03.614989 2270 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:47:03.643595 2966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:03.629822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56758 (local address 127.4.228.193:36491)
0810 18:47:03.632631 (+ 2809us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:03.632654 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:03.632692 (+ 38us) server_negotiation.cc:407] Connection header received
0810 18:47:03.632809 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:03.632819 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:03.633129 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:03.633342 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:47:03.634672 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:03.636477 (+ 1805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:03.640323 (+ 3846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:03.641991 (+ 1668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:03.642097 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:03.642276 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:03.643018 (+ 742us) server_negotiation.cc:299] Negotiation successful
0810 18:47:03.643217 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2580,"thread_start_us":2421,"threads_started":1}
I20260810 18:47:03.643594 2963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:03.629320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57402 (local address 127.4.228.195:34545)
0810 18:47:03.630127 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:03.630138 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:03.632998 (+ 2860us) server_negotiation.cc:407] Connection header received
0810 18:47:03.633344 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:03.633352 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:03.633590 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:03.633762 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:47:03.634999 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:03.636702 (+ 1703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:03.640335 (+ 3633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:03.641725 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:03.641868 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:03.642130 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:03.643012 (+ 882us) server_negotiation.cc:299] Negotiation successful
0810 18:47:03.643208 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":586,"threads_started":1}
I20260810 18:47:03.643632 2964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:03.629465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57800 (local address 127.4.228.194:38595)
0810 18:47:03.630632 (+ 1167us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:03.630642 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:03.632846 (+ 2204us) server_negotiation.cc:407] Connection header received
0810 18:47:03.633102 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:03.633112 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:03.633330 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:03.633526 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:47:03.636488 (+ 2962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:03.637892 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:03.640181 (+ 2289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:03.641763 (+ 1582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:03.641883 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:03.642077 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:03.643145 (+ 1068us) server_negotiation.cc:299] Negotiation successful
0810 18:47:03.643295 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":947,"thread_start_us":828,"threads_started":1}
I20260810 18:47:03.643649 2961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:03.629199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:36491 (local address 127.0.0.1:56758)
0810 18:47:03.632105 (+ 2906us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:03.632132 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 18:47:03.632290 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:47:03.633706 (+ 1416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:47:03.633718 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:47:03.633751 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:47:03.634400 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:03.634418 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:03.636661 (+ 2243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:03.636667 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:03.640100 (+ 3433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:03.640118 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:03.642520 (+ 2402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:03.642534 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:03.642731 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:03.642783 (+ 52us) client_negotiation.cc:769] Sending connection context
0810 18:47:03.642945 (+ 162us) client_negotiation.cc:240] Negotiation successful
0810 18:47:03.643196 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2771,"mutex_wait_us":184,"thread_start_us":3389,"threads_started":1}
I20260810 18:47:03.644459 2965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:03.628843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:38595 (local address 127.0.0.1:57800)
0810 18:47:03.631258 (+ 2415us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:03.631311 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 18:47:03.631531 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:47:03.633766 (+ 2235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:47:03.633769 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:47:03.633792 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:47:03.634751 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:03.634767 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:03.638029 (+ 3262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:03.638057 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:03.639998 (+ 1941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:03.640016 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:03.642219 (+ 2203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:03.642228 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:03.642650 (+ 422us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:03.642800 (+ 150us) client_negotiation.cc:769] Sending connection context
0810 18:47:03.642930 (+ 130us) client_negotiation.cc:240] Negotiation successful
0810 18:47:03.644087 (+ 1157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2095,"mutex_wait_us":1280,"thread_start_us":2947,"threads_started":1}
I20260810 18:47:03.645452 2962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:03.629740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:34545 (local address 127.0.0.1:57402)
0810 18:47:03.632888 (+ 3148us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:03.632908 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:47:03.633111 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:47:03.633943 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:47:03.633946 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:47:03.633974 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:47:03.634751 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:03.634767 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:03.636936 (+ 2169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:03.636943 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:03.640100 (+ 3157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:03.640118 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:03.642261 (+ 2143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:03.642266 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:03.642649 (+ 383us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:03.642802 (+ 153us) client_negotiation.cc:769] Sending connection context
0810 18:47:03.644943 (+ 2141us) client_negotiation.cc:240] Negotiation successful
0810 18:47:03.645102 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3028,"mutex_wait_us":257,"thread_start_us":1055,"threads_started":1}
I20260810 18:47:03.648023 2455 tablet_service.cc:1467] Processing CreateTablet for tablet 6d0ec30b5132476d938c4209b48bf063 (DEFAULT_TABLE table=test-table [id=0079d8442f90454abb644b7d9466e8cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:47:03.648304 2645 tablet_service.cc:1467] Processing CreateTablet for tablet 6d0ec30b5132476d938c4209b48bf063 (DEFAULT_TABLE table=test-table [id=0079d8442f90454abb644b7d9466e8cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:47:03.649473 2455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d0ec30b5132476d938c4209b48bf063. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:47:03.649616 2645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d0ec30b5132476d938c4209b48bf063. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:47:03.653548 2847 tablet_service.cc:1467] Processing CreateTablet for tablet 6d0ec30b5132476d938c4209b48bf063 (DEFAULT_TABLE table=test-table [id=0079d8442f90454abb644b7d9466e8cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:47:03.655270 2847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d0ec30b5132476d938c4209b48bf063. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:47:03.667549 2968 tablet_bootstrap.cc:492] T 6d0ec30b5132476d938c4209b48bf063 P c2dbf46fbbb64587b0d75e13af7fcc9f: Bootstrap starting.
I20260810 18:47:03.673563 2972 tablet_bootstrap.cc:492] T 6d0ec30b5132476d938c4209b48bf063 P 4c1763425d9b4cb08fabbf6d8bbe8d5a: Bootstrap starting.
I20260810 18:47:03.673791 2968 tablet_bootstrap.cc:654] T 6d0ec30b5132476d938c4209b48bf063 P c2dbf46fbbb64587b0d75e13af7fcc9f: Neither blocks nor log segments found. Creating new log.
I20260810 18:47:03.674922 2970 tablet_bootstrap.cc:492] T 6d0ec30b5132476d938c4209b48bf063 P cfda254e88174172a892a116458dcf2e: Bootstrap starting.
I20260810 18:47:03.675978 2968 log.cc:824] T 6d0ec30b5132476d938c4209b48bf063 P c2dbf46fbbb64587b0d75e13af7fcc9f: Log is configured to *not* fsync() on all Append() calls
I20260810 18:47:03.677274 2972 tablet_bootstrap.cc:654] T 6d0ec30b5132476d938c4209b48bf063 P 4c1763425d9b4cb08fabbf6d8bbe8d5a: Neither blocks nor log segments found. Creating new log.
I20260810 18:47:03.678579 2970 tablet_bootstrap.cc:654] T 6d0ec30b5132476d938c4209b48bf063 P cfda254e88174172a892a116458dcf2e: Neither blocks nor log segments found. Creating new log.
I20260810 18:47:03.678658 2972 log.cc:824] T 6d0ec30b5132476d938c4209b48bf063 P 4c1763425d9b4cb08fabbf6d8bbe8d5a: Log is configured to *not* fsync() on all Append() calls
I20260810 18:47:03.680063 2970 log.cc:824] T 6d0ec30b5132476d938c4209b48bf063 P cfda254e88174172a892a116458dcf2e: Log is configured to *not* fsync() on all Append() calls
I20260810 18:47:03.680439 2968 tablet_bootstrap.cc:492] T 6d0ec30b5132476d938c4209b48bf063 P c2dbf46fbbb64587b0d75e13af7fcc9f: No bootstrap required, opened a new log
I20260810 18:47:03.680824 2968 ts_tablet_manager.cc:1397] T 6d0ec30b5132476d938c4209b48bf063 P c2dbf46fbbb64587b0d75e13af7fcc9f: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.009s
I20260810 18:47:03.682704 2972 tablet_bootstrap.cc:492] T 6d0ec30b5132476d938c4209b48bf063 P 4c1763425d9b4cb08fabbf6d8bbe8d5a: No bootstrap required, opened a new log
I20260810 18:47:03.682994 2972 ts_tablet_manager.cc:1397] T 6d0ec30b5132476d938c4209b48bf063 P 4c1763425d9b4cb08fabbf6d8bbe8d5a: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260810 18:47:03.685097 2970 tablet_bootstrap.cc:492] T 6d0ec30b5132476d938c4209b48bf063 P cfda254e88174172a892a116458dcf2e: No bootstrap required, opened a new log
I20260810 18:47:03.685402 2970 ts_tablet_manager.cc:1397] T 6d0ec30b5132476d938c4209b48bf063 P cfda254e88174172a892a116458dcf2e: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260810 18:47:03.691919 2968 raft_consensus.cc:348] T 6d0ec30b5132476d938c4209b48bf063 P c2dbf46fbbb64587b0d75e13af7fcc9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfda254e88174172a892a116458dcf2e" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 38595 } } peers { permanent_uuid: "4c1763425d9b4cb08fabbf6d8bbe8d5a" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 34545 } } peers { permanent_uuid: "c2dbf46fbbb64587b0d75e13af7fcc9f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 36491 } }
I20260810 18:47:03.692428 2968 raft_consensus.cc:374] T 6d0ec30b5132476d938c4209b48bf063 P c2dbf46fbbb64587b0d75e13af7fcc9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:47:03.692525 2968 raft_consensus.cc:729] T 6d0ec30b5132476d938c4209b48bf063 P c2dbf46fbbb64587b0d75e13af7fcc9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2dbf46fbbb64587b0d75e13af7fcc9f, State: Initialized, Role: FOLLOWER
I20260810 18:47:03.693780 2968 consensus_queue.cc:260] T 6d0ec30b5132476d938c4209b48bf063 P c2dbf46fbbb64587b0d75e13af7fcc9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfda254e88174172a892a116458dcf2e" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 38595 } } peers { permanent_uuid: "4c1763425d9b4cb08fabbf6d8bbe8d5a" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 34545 } } peers { permanent_uuid: "c2dbf46fbbb64587b0d75e13af7fcc9f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 36491 } }
I20260810 18:47:03.693682 2970 raft_consensus.cc:348] T 6d0ec30b5132476d938c4209b48bf063 P cfda254e88174172a892a116458dcf2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfda254e88174172a892a116458dcf2e" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 38595 } } peers { permanent_uuid: "4c1763425d9b4cb08fabbf6d8bbe8d5a" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 34545 } } peers { permanent_uuid: "c2dbf46fbbb64587b0d75e13af7fcc9f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 36491 } }
I20260810 18:47:03.693747 2972 raft_consensus.cc:348] T 6d0ec30b5132476d938c4209b48bf063 P 4c1763425d9b4cb08fabbf6d8bbe8d5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfda254e88174172a892a116458dcf2e" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 38595 } } peers { permanent_uuid: "4c1763425d9b4cb08fabbf6d8bbe8d5a" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 34545 } } peers { permanent_uuid: "c2dbf46fbbb64587b0d75e13af7fcc9f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 36491 } }
I20260810 18:47:03.694144 2970 raft_consensus.cc:374] T 6d0ec30b5132476d938c4209b48bf063 P cfda254e88174172a892a116458dcf2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:47:03.694159 2972 raft_consensus.cc:374] T 6d0ec30b5132476d938c4209b48bf063 P 4c1763425d9b4cb08fabbf6d8bbe8d5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:47:03.694257 2970 raft_consensus.cc:729] T 6d0ec30b5132476d938c4209b48bf063 P cfda254e88174172a892a116458dcf2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfda254e88174172a892a116458dcf2e, State: Initialized, Role: FOLLOWER
I20260810 18:47:03.694262 2972 raft_consensus.cc:729] T 6d0ec30b5132476d938c4209b48bf063 P 4c1763425d9b4cb08fabbf6d8bbe8d5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c1763425d9b4cb08fabbf6d8bbe8d5a, State: Initialized, Role: FOLLOWER
I20260810 18:47:03.694947 2972 consensus_queue.cc:260] T 6d0ec30b5132476d938c4209b48bf063 P 4c1763425d9b4cb08fabbf6d8bbe8d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfda254e88174172a892a116458dcf2e" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 38595 } } peers { permanent_uuid: "4c1763425d9b4cb08fabbf6d8bbe8d5a" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 34545 } } peers { permanent_uuid: "c2dbf46fbbb64587b0d75e13af7fcc9f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 36491 } }
I20260810 18:47:03.695039 2970 consensus_queue.cc:260] T 6d0ec30b5132476d938c4209b48bf063 P cfda254e88174172a892a116458dcf2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfda254e88174172a892a116458dcf2e" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 38595 } } peers { permanent_uuid: "4c1763425d9b4cb08fabbf6d8bbe8d5a" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 34545 } } peers { permanent_uuid: "c2dbf46fbbb64587b0d75e13af7fcc9f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 36491 } }
I20260810 18:47:03.696041 2968 ts_tablet_manager.cc:1428] T 6d0ec30b5132476d938c4209b48bf063 P c2dbf46fbbb64587b0d75e13af7fcc9f: Time spent starting tablet: real 0.015s user 0.000s sys 0.014s
I20260810 18:47:03.697551 2744 heartbeater.cc:503] Master 127.4.228.254:40343 was elected leader, sending a full tablet report...
I20260810 18:47:03.697649 2972 ts_tablet_manager.cc:1428] T 6d0ec30b5132476d938c4209b48bf063 P 4c1763425d9b4cb08fabbf6d8bbe8d5a: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260810 18:47:03.698222 2970 ts_tablet_manager.cc:1428] T 6d0ec30b5132476d938c4209b48bf063 P cfda254e88174172a892a116458dcf2e: Time spent starting tablet: real 0.013s user 0.013s sys 0.001s
I20260810 18:47:03.698565 2942 heartbeater.cc:503] Master 127.4.228.254:40343 was elected leader, sending a full tablet report...
W20260810 18:47:03.716717 2745 tablet.cc:2369] T 6d0ec30b5132476d938c4209b48bf063 P cfda254e88174172a892a116458dcf2e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:47:03.730921 2984 raft_consensus.cc:482] T 6d0ec30b5132476d938c4209b48bf063 P cfda254e88174172a892a116458dcf2e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:47:03.731220 2984 raft_consensus.cc:504] T 6d0ec30b5132476d938c4209b48bf063 P cfda254e88174172a892a116458dcf2e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfda254e88174172a892a116458dcf2e" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 38595 } } peers { permanent_uuid: "4c1763425d9b4cb08fabbf6d8bbe8d5a" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 34545 } } peers { permanent_uuid: "c2dbf46fbbb64587b0d75e13af7fcc9f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 36491 } }
I20260810 18:47:03.733011 2984 leader_election.cc:288] T 6d0ec30b5132476d938c4209b48bf063 P cfda254e88174172a892a116458dcf2e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c1763425d9b4cb08fabbf6d8bbe8d5a (127.4.228.195:34545), c2dbf46fbbb64587b0d75e13af7fcc9f (127.4.228.193:36491)
I20260810 18:47:03.741557 2966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:03.733514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:41771 (local address 127.4.228.193:36491)
0810 18:47:03.733715 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:03.733723 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:03.734878 (+ 1155us) server_negotiation.cc:407] Connection header received
0810 18:47:03.735121 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:03.735126 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:03.735210 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:03.735281 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:47:03.735920 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:03.736952 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:03.739164 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:03.740319 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:03.740458 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:03.740535 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:03.741152 (+ 617us) server_negotiation.cc:299] Negotiation successful
0810 18:47:03.741290 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 18:47:03.742210 2963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:03.733399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:38949 (local address 127.4.228.195:34545)
0810 18:47:03.733622 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:03.733629 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:03.734446 (+ 817us) server_negotiation.cc:407] Connection header received
0810 18:47:03.734793 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:03.734801 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:03.734902 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:03.735027 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:47:03.735708 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:03.736904 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:03.739751 (+ 2847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:03.740990 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:03.741150 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:03.741232 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:03.741841 (+ 609us) server_negotiation.cc:299] Negotiation successful
0810 18:47:03.741961 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 18:47:03.742290 2988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:03.733348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:36491 (local address 127.4.228.194:41771)
0810 18:47:03.734774 (+ 1426us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:03.734788 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:47:03.734918 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:47:03.735335 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:47:03.735347 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:47:03.735370 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:47:03.735777 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:03.735787 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:03.737124 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:03.737128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:03.738983 (+ 1855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:03.738998 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:03.740757 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:03.740764 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:03.740932 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:03.740969 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 18:47:03.741761 (+ 792us) client_negotiation.cc:240] Negotiation successful
0810 18:47:03.741901 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1306,"mutex_wait_us":46,"thread_start_us":1041,"threads_started":1}
I20260810 18:47:03.742863 2985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:03.733347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:34545 (local address 127.4.228.194:38949)
0810 18:47:03.734303 (+ 956us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:03.734331 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 18:47:03.734529 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:47:03.735049 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:47:03.735054 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:47:03.735078 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:47:03.735518 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:03.735535 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:03.737084 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:03.737106 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:03.739584 (+ 2478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:03.739598 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:03.741374 (+ 1776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:03.741381 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:03.741553 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:03.741596 (+ 43us) client_negotiation.cc:769] Sending connection context
0810 18:47:03.742402 (+ 806us) client_negotiation.cc:240] Negotiation successful
0810 18:47:03.742566 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":802,"mutex_wait_us":1,"thread_start_us":495,"threads_started":1}
I20260810 18:47:03.743324 2480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d0ec30b5132476d938c4209b48bf063" candidate_uuid: "cfda254e88174172a892a116458dcf2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2dbf46fbbb64587b0d75e13af7fcc9f" is_pre_election: true
I20260810 18:47:03.743872 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d0ec30b5132476d938c4209b48bf063" candidate_uuid: "cfda254e88174172a892a116458dcf2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c1763425d9b4cb08fabbf6d8bbe8d5a" is_pre_election: true
I20260810 18:47:03.744002 2480 raft_consensus.cc:2454] T 6d0ec30b5132476d938c4209b48bf063 P c2dbf46fbbb64587b0d75e13af7fcc9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cfda254e88174172a892a116458dcf2e in term 0.
I20260810 18:47:03.744460 2877 raft_consensus.cc:2454] T 6d0ec30b5132476d938c4209b48bf063 P 4c1763425d9b4cb08fabbf6d8bbe8d5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cfda254e88174172a892a116458dcf2e in term 0.
I20260810 18:47:03.744830 2588 leader_election.cc:302] T 6d0ec30b5132476d938c4209b48bf063 P cfda254e88174172a892a116458dcf2e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c2dbf46fbbb64587b0d75e13af7fcc9f, cfda254e88174172a892a116458dcf2e; no voters:
I20260810 18:47:03.745250 2984 raft_consensus.cc:2781] T 6d0ec30b5132476d938c4209b48bf063 P cfda254e88174172a892a116458dcf2e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:47:03.745424 2984 raft_consensus.cc:482] T 6d0ec30b5132476d938c4209b48bf063 P cfda254e88174172a892a116458dcf2e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:47:03.745534 2984 raft_consensus.cc:3037] T 6d0ec30b5132476d938c4209b48bf063 P cfda254e88174172a892a116458dcf2e [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:47:03.747946 2984 raft_consensus.cc:504] T 6d0ec30b5132476d938c4209b48bf063 P cfda254e88174172a892a116458dcf2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfda254e88174172a892a116458dcf2e" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 38595 } } peers { permanent_uuid: "4c1763425d9b4cb08fabbf6d8bbe8d5a" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 34545 } } peers { permanent_uuid: "c2dbf46fbbb64587b0d75e13af7fcc9f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 36491 } }
I20260810 18:47:03.748538 2984 leader_election.cc:288] T 6d0ec30b5132476d938c4209b48bf063 P cfda254e88174172a892a116458dcf2e [CANDIDATE]: Term 1 election: Requested vote from peers 4c1763425d9b4cb08fabbf6d8bbe8d5a (127.4.228.195:34545), c2dbf46fbbb64587b0d75e13af7fcc9f (127.4.228.193:36491)
I20260810 18:47:03.749001 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d0ec30b5132476d938c4209b48bf063" candidate_uuid: "cfda254e88174172a892a116458dcf2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c1763425d9b4cb08fabbf6d8bbe8d5a"
I20260810 18:47:03.749025 2480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d0ec30b5132476d938c4209b48bf063" candidate_uuid: "cfda254e88174172a892a116458dcf2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2dbf46fbbb64587b0d75e13af7fcc9f"
I20260810 18:47:03.749271 2877 raft_consensus.cc:3037] T 6d0ec30b5132476d938c4209b48bf063 P 4c1763425d9b4cb08fabbf6d8bbe8d5a [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:47:03.749323 2480 raft_consensus.cc:3037] T 6d0ec30b5132476d938c4209b48bf063 P c2dbf46fbbb64587b0d75e13af7fcc9f [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:47:03.751750 2877 raft_consensus.cc:2454] T 6d0ec30b5132476d938c4209b48bf063 P 4c1763425d9b4cb08fabbf6d8bbe8d5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cfda254e88174172a892a116458dcf2e in term 1.
I20260810 18:47:03.751802 2480 raft_consensus.cc:2454] T 6d0ec30b5132476d938c4209b48bf063 P c2dbf46fbbb64587b0d75e13af7fcc9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cfda254e88174172a892a116458dcf2e in term 1.
I20260810 18:47:03.752187 2585 leader_election.cc:302] T 6d0ec30b5132476d938c4209b48bf063 P cfda254e88174172a892a116458dcf2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c1763425d9b4cb08fabbf6d8bbe8d5a, cfda254e88174172a892a116458dcf2e; no voters:
I20260810 18:47:03.752561 2984 raft_consensus.cc:2781] T 6d0ec30b5132476d938c4209b48bf063 P cfda254e88174172a892a116458dcf2e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:47:03.753125 2984 raft_consensus.cc:686] T 6d0ec30b5132476d938c4209b48bf063 P cfda254e88174172a892a116458dcf2e [term 1 LEADER]: Becoming Leader. State: Replica: cfda254e88174172a892a116458dcf2e, State: Running, Role: LEADER
I20260810 18:47:03.753733 2984 consensus_queue.cc:237] T 6d0ec30b5132476d938c4209b48bf063 P cfda254e88174172a892a116458dcf2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfda254e88174172a892a116458dcf2e" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 38595 } } peers { permanent_uuid: "4c1763425d9b4cb08fabbf6d8bbe8d5a" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 34545 } } peers { permanent_uuid: "c2dbf46fbbb64587b0d75e13af7fcc9f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 36491 } }
I20260810 18:47:03.759057 2270 catalog_manager.cc:5495] T 6d0ec30b5132476d938c4209b48bf063 P cfda254e88174172a892a116458dcf2e reported cstate change: term changed from 0 to 1, leader changed from <none> to cfda254e88174172a892a116458dcf2e (127.4.228.194). New cstate: current_term: 1 leader_uuid: "cfda254e88174172a892a116458dcf2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfda254e88174172a892a116458dcf2e" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 38595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c1763425d9b4cb08fabbf6d8bbe8d5a" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 34545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2dbf46fbbb64587b0d75e13af7fcc9f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 36491 } health_report { overall_health: UNKNOWN } } }
I20260810 18:47:03.796247 2964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:03.790684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57804 (local address 127.4.228.194:38595)
0810 18:47:03.790911 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:03.790920 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:03.790984 (+ 64us) server_negotiation.cc:407] Connection header received
0810 18:47:03.791270 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:03.791274 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:03.791356 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:03.791519 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:47:03.791925 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:03.792682 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:03.794040 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:03.794452 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:03.794650 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:03.795121 (+ 471us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:47:03.795638 (+ 517us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:47:03.795821 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:03.795870 (+ 49us) server_negotiation.cc:299] Negotiation successful
0810 18:47:03.795954 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260810 18:47:03.797304 2944 tablet.cc:2369] T 6d0ec30b5132476d938c4209b48bf063 P 4c1763425d9b4cb08fabbf6d8bbe8d5a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:47:03.804817 2480 raft_consensus.cc:1261] T 6d0ec30b5132476d938c4209b48bf063 P c2dbf46fbbb64587b0d75e13af7fcc9f [term 1 FOLLOWER]: Refusing update from remote peer cfda254e88174172a892a116458dcf2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:47:03.804849 2877 raft_consensus.cc:1261] T 6d0ec30b5132476d938c4209b48bf063 P 4c1763425d9b4cb08fabbf6d8bbe8d5a [term 1 FOLLOWER]: Refusing update from remote peer cfda254e88174172a892a116458dcf2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:47:03.805905 2990 consensus_queue.cc:1035] T 6d0ec30b5132476d938c4209b48bf063 P cfda254e88174172a892a116458dcf2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c1763425d9b4cb08fabbf6d8bbe8d5a" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 34545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:47:03.806294 2984 consensus_queue.cc:1035] T 6d0ec30b5132476d938c4209b48bf063 P cfda254e88174172a892a116458dcf2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2dbf46fbbb64587b0d75e13af7fcc9f" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 36491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:47:03.833757 2269 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:47:03.834123 2269 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:47:03.838523 2269 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6579ad59ed4a44449947bce31c1f463f: Sending DeleteTablet for 3 replicas of tablet 6d0ec30b5132476d938c4209b48bf063
I20260810 18:47:03.839985 2645 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d0ec30b5132476d938c4209b48bf063 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:47:03 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:57800
I20260810 18:47:03.840107 2847 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d0ec30b5132476d938c4209b48bf063 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:47:03 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:57402
I20260810 18:47:03.840267 2455 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d0ec30b5132476d938c4209b48bf063 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:47:03 UTC) from {username='kudu', principal='kudu/127.4.228.254@KRBTEST.COM'} at 127.0.0.1:56758
I20260810 18:47:03.842343 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 2364
I20260810 18:47:03.852990 3008 tablet_replica.cc:330] T 6d0ec30b5132476d938c4209b48bf063 P 4c1763425d9b4cb08fabbf6d8bbe8d5a: stopping tablet replica
I20260810 18:47:03.853572 3008 raft_consensus.cc:2229] T 6d0ec30b5132476d938c4209b48bf063 P 4c1763425d9b4cb08fabbf6d8bbe8d5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:47:03.854234 3008 raft_consensus.cc:2258] T 6d0ec30b5132476d938c4209b48bf063 P 4c1763425d9b4cb08fabbf6d8bbe8d5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:47:03.856922 3008 ts_tablet_manager.cc:1905] T 6d0ec30b5132476d938c4209b48bf063 P 4c1763425d9b4cb08fabbf6d8bbe8d5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:47:03.865895 3008 ts_tablet_manager.cc:1918] T 6d0ec30b5132476d938c4209b48bf063 P 4c1763425d9b4cb08fabbf6d8bbe8d5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:47:03.866780 3009 tablet_replica.cc:330] T 6d0ec30b5132476d938c4209b48bf063 P c2dbf46fbbb64587b0d75e13af7fcc9f: stopping tablet replica
I20260810 18:47:03.866994 3008 log.cc:1196] T 6d0ec30b5132476d938c4209b48bf063 P 4c1763425d9b4cb08fabbf6d8bbe8d5a: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-2/wal/wals/6d0ec30b5132476d938c4209b48bf063
I20260810 18:47:03.867405 3009 raft_consensus.cc:2229] T 6d0ec30b5132476d938c4209b48bf063 P c2dbf46fbbb64587b0d75e13af7fcc9f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:47:03.867844 3008 ts_tablet_manager.cc:1939] T 6d0ec30b5132476d938c4209b48bf063 P 4c1763425d9b4cb08fabbf6d8bbe8d5a: Deleting consensus metadata
I20260810 18:47:03.868057 3009 raft_consensus.cc:2258] T 6d0ec30b5132476d938c4209b48bf063 P c2dbf46fbbb64587b0d75e13af7fcc9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:47:03.869851 3006 tablet_replica.cc:330] T 6d0ec30b5132476d938c4209b48bf063 P cfda254e88174172a892a116458dcf2e: stopping tablet replica
I20260810 18:47:03.870237 2247 catalog_manager.cc:4841] TS 4c1763425d9b4cb08fabbf6d8bbe8d5a (127.4.228.195:34545): tablet 6d0ec30b5132476d938c4209b48bf063 (table test-table [id=0079d8442f90454abb644b7d9466e8cb]) successfully deleted
I20260810 18:47:03.870357 3006 raft_consensus.cc:2229] T 6d0ec30b5132476d938c4209b48bf063 P cfda254e88174172a892a116458dcf2e [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:47:03.871182 3006 raft_consensus.cc:2258] T 6d0ec30b5132476d938c4209b48bf063 P cfda254e88174172a892a116458dcf2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:47:03.871865 3009 ts_tablet_manager.cc:1905] T 6d0ec30b5132476d938c4209b48bf063 P c2dbf46fbbb64587b0d75e13af7fcc9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:47:03.873471 3006 ts_tablet_manager.cc:1905] T 6d0ec30b5132476d938c4209b48bf063 P cfda254e88174172a892a116458dcf2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:47:03.879280 3009 ts_tablet_manager.cc:1918] T 6d0ec30b5132476d938c4209b48bf063 P c2dbf46fbbb64587b0d75e13af7fcc9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:47:03.879527 3009 log.cc:1196] T 6d0ec30b5132476d938c4209b48bf063 P c2dbf46fbbb64587b0d75e13af7fcc9f: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-0/wal/wals/6d0ec30b5132476d938c4209b48bf063
I20260810 18:47:03.880178 3009 ts_tablet_manager.cc:1939] T 6d0ec30b5132476d938c4209b48bf063 P c2dbf46fbbb64587b0d75e13af7fcc9f: Deleting consensus metadata
I20260810 18:47:03.881592 3006 ts_tablet_manager.cc:1918] T 6d0ec30b5132476d938c4209b48bf063 P cfda254e88174172a892a116458dcf2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:47:03.881810 3006 log.cc:1196] T 6d0ec30b5132476d938c4209b48bf063 P cfda254e88174172a892a116458dcf2e: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387267874959-5011-0/minicluster-data/ts-1/wal/wals/6d0ec30b5132476d938c4209b48bf063
I20260810 18:47:03.882341 2252 catalog_manager.cc:4841] TS c2dbf46fbbb64587b0d75e13af7fcc9f (127.4.228.193:36491): tablet 6d0ec30b5132476d938c4209b48bf063 (table test-table [id=0079d8442f90454abb644b7d9466e8cb]) successfully deleted
I20260810 18:47:03.882453 3006 ts_tablet_manager.cc:1939] T 6d0ec30b5132476d938c4209b48bf063 P cfda254e88174172a892a116458dcf2e: Deleting consensus metadata
Aug 10 18:47:03 dist-test-slave-7kzw krb5kdc[2203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.193@KRBTEST.COM
I20260810 18:47:03.884439 2251 catalog_manager.cc:4841] TS cfda254e88174172a892a116458dcf2e (127.4.228.194:38595): tablet 6d0ec30b5132476d938c4209b48bf063 (table test-table [id=0079d8442f90454abb644b7d9466e8cb]) successfully deleted
I20260810 18:47:03.891712 2966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:03.846853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56774 (local address 127.4.228.193:36491)
0810 18:47:03.847124 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:03.847135 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:03.869024 (+ 21889us) server_negotiation.cc:407] Connection header received
0810 18:47:03.869593 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:03.869602 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:03.869719 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:03.869844 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:03.874094 (+ 4250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:03.875354 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:03.877555 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:03.878245 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:03.878363 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:03.884798 (+ 6435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:03.884812 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:03.884828 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:47:03.884872 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:03.888098 (+ 3226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:03.889203 (+ 1105us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:03.889211 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:03.889214 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:03.889381 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:03.889859 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:03.889864 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:03.889866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:03.890292 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:03.890667 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:03.891057 (+ 390us) server_negotiation.cc:299] Negotiation successful
0810 18:47:03.891320 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260810 18:47:03.892557 2544 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:47:03.997579 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 2364
I20260810 18:47:04.029189 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 2567
Aug 10 18:47:04 dist-test-slave-7kzw krb5kdc[2203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.194@KRBTEST.COM
I20260810 18:47:04.049366 2964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:04.030185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57812 (local address 127.4.228.194:38595)
0810 18:47:04.030428 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:04.030433 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:04.030455 (+ 22us) server_negotiation.cc:407] Connection header received
0810 18:47:04.030769 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:04.030774 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:04.030863 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:04.030958 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:04.032921 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:04.033986 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:04.036286 (+ 2300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:04.036810 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:04.036951 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:04.042866 (+ 5915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:04.042881 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:04.042900 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:47:04.042942 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:04.046305 (+ 3363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:04.047050 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:04.047059 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:04.047061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:04.047217 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:04.047643 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:04.047647 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:04.047649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:04.048031 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:04.048437 (+ 406us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:04.048744 (+ 307us) server_negotiation.cc:299] Negotiation successful
0810 18:47:04.048970 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260810 18:47:04.050078 2736 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:47:04.166656 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 2567
I20260810 18:47:04.196239 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 2750
Aug 10 18:47:04 dist-test-slave-7kzw krb5kdc[2203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.195@KRBTEST.COM
I20260810 18:47:04.213197 2963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:04.197146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57406 (local address 127.4.228.195:34545)
0810 18:47:04.197345 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:04.197354 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:04.197378 (+ 24us) server_negotiation.cc:407] Connection header received
0810 18:47:04.197599 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:04.197609 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:04.197695 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:04.197812 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:04.199610 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:04.200326 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:04.201632 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:04.202036 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:04.202181 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:04.207358 (+ 5177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:04.207374 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:04.207394 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:47:04.207432 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:04.210364 (+ 2932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:04.211189 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:04.211197 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:04.211200 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:04.211354 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:04.211811 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:04.211815 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:04.211818 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:04.212209 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:04.212454 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:04.212761 (+ 307us) server_negotiation.cc:299] Negotiation successful
0810 18:47:04.212929 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 18:47:04.213905 2936 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:47:04.313498 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 2750
I20260810 18:47:04.344478 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 2226
I20260810 18:47:04.345724 2312 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:47:04.458277 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 2226
2026-08-10T18:47:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4942 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-10T18:47:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:47:04Z Disabled control of system clock
I20260810 18:47:04.564131 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:38449
--webserver_interface=127.4.228.254
--webserver_port=0
--builtin_ntp_servers=127.4.228.212:35741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:38449
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 18:47:04.927385 3027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:47:04.927704 3027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:47:04.927790 3027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:47:04.935153 3027 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:47:04.935264 3027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:47:04.935302 3027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:47:04.935362 3027 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:47:04.935408 3027 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:47:04.944841 3027 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:35741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:38449
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:38449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:47:04.946218 3027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:47:04.948308 3027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:47:04.960915 3034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:04.961014 3039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:04.961061 3036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:47:04.962270 3027 server_base.cc:1034] running on GCE node
I20260810 18:47:04.963235 3027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:47:04.964726 3027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:47:04.965961 3027 hybrid_clock.cc:648] HybridClock initialized: now 1786387624965905 us; error 66 us; skew 500 ppm
I20260810 18:47:04.966356 3027 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:47:04.968801 3027 webserver.cc:460] Webserver started at http://127.4.228.254:37069/ using document root <none> and password file <none>
I20260810 18:47:04.969543 3027 fs_manager.cc:359] Metadata directory not provided
I20260810 18:47:04.969679 3027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:47:04.970053 3027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:47:04.973124 3027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "13d3829da4f04ab19f67f4dbd3fd0d93"
format_stamp: "Formatted at 2026-08-10 18:47:04 on dist-test-slave-7kzw"
server_key: "d773e6d88439305958c7c3f7bc8a178e"
server_key_iv: "61bfc5548e33064dc4ce53c5739d0c13"
server_key_version: "encryptionkey@0"
I20260810 18:47:04.973973 3027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "13d3829da4f04ab19f67f4dbd3fd0d93"
format_stamp: "Formatted at 2026-08-10 18:47:04 on dist-test-slave-7kzw"
server_key: "d773e6d88439305958c7c3f7bc8a178e"
server_key_iv: "61bfc5548e33064dc4ce53c5739d0c13"
server_key_version: "encryptionkey@0"
I20260810 18:47:04.979444 3027 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 18:47:04.983033 3046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:04.984426 3027 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 18:47:04.984632 3027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "13d3829da4f04ab19f67f4dbd3fd0d93"
format_stamp: "Formatted at 2026-08-10 18:47:04 on dist-test-slave-7kzw"
server_key: "d773e6d88439305958c7c3f7bc8a178e"
server_key_iv: "61bfc5548e33064dc4ce53c5739d0c13"
server_key_version: "encryptionkey@0"
I20260810 18:47:04.984858 3027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:47:05.001808 3027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:47:05.002779 3027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:47:05.003096 3027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:47:05.023684 3027 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:38449
I20260810 18:47:05.023756 3111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:38449 every 8 connection(s)
I20260810 18:47:05.025136 3027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:47:05.026443 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 3027
I20260810 18:47:05.026714 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:47:05.027297 5011 external_mini_cluster.cc:1428] Setting key fd59ccf2ae131a7372ede9dd96a03da4
I20260810 18:47:05.030912 3112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:47:05.042027 3120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:05.029193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35668 (local address 127.4.228.254:38449)
0810 18:47:05.030240 (+ 1047us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:05.030252 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:05.030296 (+ 44us) server_negotiation.cc:407] Connection header received
0810 18:47:05.032060 (+ 1764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:05.032084 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:05.032539 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:05.032860 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:05.033588 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:05.035065 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:05.037023 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:05.037730 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:05.037873 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:05.039320 (+ 1447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:05.039329 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:05.039344 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:47:05.039947 (+ 603us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:05.040029 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:47:05.040058 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:05.040207 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:05.040460 (+ 253us) server_negotiation.cc:299] Negotiation successful
0810 18:47:05.040870 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":398,"threads_started":1}
I20260810 18:47:05.044466 3112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13d3829da4f04ab19f67f4dbd3fd0d93: Bootstrap starting.
I20260810 18:47:05.047945 3112 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13d3829da4f04ab19f67f4dbd3fd0d93: Neither blocks nor log segments found. Creating new log.
I20260810 18:47:05.049294 3112 log.cc:824] T 00000000000000000000000000000000 P 13d3829da4f04ab19f67f4dbd3fd0d93: Log is configured to *not* fsync() on all Append() calls
I20260810 18:47:05.052685 3112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13d3829da4f04ab19f67f4dbd3fd0d93: No bootstrap required, opened a new log
I20260810 18:47:05.061093 3112 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13d3829da4f04ab19f67f4dbd3fd0d93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13d3829da4f04ab19f67f4dbd3fd0d93" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 38449 } }
I20260810 18:47:05.061482 3112 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13d3829da4f04ab19f67f4dbd3fd0d93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:47:05.061566 3112 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13d3829da4f04ab19f67f4dbd3fd0d93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13d3829da4f04ab19f67f4dbd3fd0d93, State: Initialized, Role: FOLLOWER
I20260810 18:47:05.062362 3112 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13d3829da4f04ab19f67f4dbd3fd0d93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13d3829da4f04ab19f67f4dbd3fd0d93" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 38449 } }
I20260810 18:47:05.062690 3112 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13d3829da4f04ab19f67f4dbd3fd0d93 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:47:05.062844 3112 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13d3829da4f04ab19f67f4dbd3fd0d93 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:47:05.063074 3112 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13d3829da4f04ab19f67f4dbd3fd0d93 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:47:05.065999 3112 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13d3829da4f04ab19f67f4dbd3fd0d93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13d3829da4f04ab19f67f4dbd3fd0d93" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 38449 } }
I20260810 18:47:05.066656 3112 leader_election.cc:302] T 00000000000000000000000000000000 P 13d3829da4f04ab19f67f4dbd3fd0d93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13d3829da4f04ab19f67f4dbd3fd0d93; no voters:
I20260810 18:47:05.067324 3112 leader_election.cc:288] T 00000000000000000000000000000000 P 13d3829da4f04ab19f67f4dbd3fd0d93 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:47:05.067487 3122 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13d3829da4f04ab19f67f4dbd3fd0d93 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:47:05.068233 3122 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13d3829da4f04ab19f67f4dbd3fd0d93 [term 1 LEADER]: Becoming Leader. State: Replica: 13d3829da4f04ab19f67f4dbd3fd0d93, State: Running, Role: LEADER
I20260810 18:47:05.068744 3122 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13d3829da4f04ab19f67f4dbd3fd0d93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13d3829da4f04ab19f67f4dbd3fd0d93" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 38449 } }
I20260810 18:47:05.069641 3112 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13d3829da4f04ab19f67f4dbd3fd0d93 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:47:05.072196 3124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13d3829da4f04ab19f67f4dbd3fd0d93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13d3829da4f04ab19f67f4dbd3fd0d93. Latest consensus state: current_term: 1 leader_uuid: "13d3829da4f04ab19f67f4dbd3fd0d93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13d3829da4f04ab19f67f4dbd3fd0d93" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 38449 } } }
I20260810 18:47:05.072713 3124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13d3829da4f04ab19f67f4dbd3fd0d93 [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:05.073498 3123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13d3829da4f04ab19f67f4dbd3fd0d93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13d3829da4f04ab19f67f4dbd3fd0d93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13d3829da4f04ab19f67f4dbd3fd0d93" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 38449 } } }
I20260810 18:47:05.073746 3133 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:47:05.073822 3123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13d3829da4f04ab19f67f4dbd3fd0d93 [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:05.079663 3133 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:47:05.089375 3133 catalog_manager.cc:1347] Generated new cluster ID: b7696b27df4e4733bee37d9246ad5ed7
I20260810 18:47:05.089548 3133 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:47:05.147260 3133 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:47:05.148901 3133 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:47:05.156737 3133 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13d3829da4f04ab19f67f4dbd3fd0d93: Generated new TSK 0
I20260810 18:47:05.157596 3133 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:47:05.163030 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.193:0
--local_ip_for_outbound_sockets=127.4.228.193
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:38449
--builtin_ntp_servers=127.4.228.212:35741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 18:47:05.532846 3154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:47:05.533172 3154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:47:05.533335 3154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:47:05.540699 3154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:47:05.540868 3154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:47:05.541108 3154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.193
I20260810 18:47:05.550964 3154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:35741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.228.193
--webserver_port=0
--tserver_master_addrs=127.4.228.254:38449
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.193
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:47:05.552423 3154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:47:05.554353 3154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:47:05.569403 3163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:05.569451 3161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:05.569499 3160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:47:05.575234 3154 server_base.cc:1034] running on GCE node
I20260810 18:47:05.575941 3154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:47:05.576931 3154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:47:05.578181 3154 hybrid_clock.cc:648] HybridClock initialized: now 1786387625578131 us; error 43 us; skew 500 ppm
I20260810 18:47:05.578601 3154 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:47:05.581017 3154 webserver.cc:460] Webserver started at http://127.4.228.193:33105/ using document root <none> and password file <none>
I20260810 18:47:05.581751 3154 fs_manager.cc:359] Metadata directory not provided
I20260810 18:47:05.581910 3154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:47:05.582219 3154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:47:05.585170 3154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-0/data/instance:
uuid: "f5999bdb0a8942e5a0917cf322d8c766"
format_stamp: "Formatted at 2026-08-10 18:47:05 on dist-test-slave-7kzw"
server_key: "08637cad9c76779bf32775bf37abb868"
server_key_iv: "b80cb2cb998a7836cea93a85f090f9de"
server_key_version: "encryptionkey@0"
I20260810 18:47:05.585923 3154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance:
uuid: "f5999bdb0a8942e5a0917cf322d8c766"
format_stamp: "Formatted at 2026-08-10 18:47:05 on dist-test-slave-7kzw"
server_key: "08637cad9c76779bf32775bf37abb868"
server_key_iv: "b80cb2cb998a7836cea93a85f090f9de"
server_key_version: "encryptionkey@0"
I20260810 18:47:05.591233 3154 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 18:47:05.595479 3169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:05.597167 3154 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:47:05.597373 3154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-0/wal
uuid: "f5999bdb0a8942e5a0917cf322d8c766"
format_stamp: "Formatted at 2026-08-10 18:47:05 on dist-test-slave-7kzw"
server_key: "08637cad9c76779bf32775bf37abb868"
server_key_iv: "b80cb2cb998a7836cea93a85f090f9de"
server_key_version: "encryptionkey@0"
I20260810 18:47:05.597610 3154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:47:05.642711 3154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:47:05.643669 3154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:47:05.644009 3154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:47:05.645154 3154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:47:05.646960 3154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:47:05.647094 3154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:05.647240 3154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:47:05.647333 3154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:05.695222 3154 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.193:43797
I20260810 18:47:05.695327 3346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.193:43797 every 8 connection(s)
I20260810 18:47:05.696612 3154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-0/data/info.pb
I20260810 18:47:05.704456 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 3154
I20260810 18:47:05.704653 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-0/wal/instance
I20260810 18:47:05.705247 5011 external_mini_cluster.cc:1428] Setting key 22495687b65c5db1d90d5f951d819242
I20260810 18:47:05.708783 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.194:0
--local_ip_for_outbound_sockets=127.4.228.194
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:38449
--builtin_ntp_servers=127.4.228.212:35741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 18:47:05.711666 3351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:05.699431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:36871 (local address 127.4.228.254:38449)
0810 18:47:05.699914 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:05.699919 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:05.701275 (+ 1356us) server_negotiation.cc:407] Connection header received
0810 18:47:05.703365 (+ 2090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:05.703371 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:05.703475 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:05.703602 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:05.704794 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:05.705864 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:05.708089 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:05.708851 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:05.709056 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:05.710561 (+ 1505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:05.710571 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:05.710581 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:47:05.710611 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:05.710673 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:47:05.710696 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:05.710806 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:05.711028 (+ 222us) server_negotiation.cc:299] Negotiation successful
0810 18:47:05.711160 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":252,"threads_started":1}
I20260810 18:47:05.712491 3354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:05.700087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:38449 (local address 127.4.228.193:36871)
0810 18:47:05.701081 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:05.701121 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 18:47:05.703076 (+ 1955us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:47:05.703862 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:47:05.703869 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:47:05.703959 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:47:05.704537 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:05.704554 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:05.706064 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:05.706073 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:05.707912 (+ 1839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:05.707926 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:05.709183 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:05.709189 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:05.709396 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:05.710179 (+ 783us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:47:05.710182 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:47:05.710228 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:47:05.710310 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:47:05.710804 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:47:05.710814 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:47:05.710821 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 18:47:05.710971 (+ 150us) client_negotiation.cc:240] Negotiation successful
0810 18:47:05.711271 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":292,"threads_started":1}
I20260810 18:47:05.714385 3349 heartbeater.cc:347] Connected to a master server at 127.4.228.254:38449
I20260810 18:47:05.714720 3349 heartbeater.cc:464] Registering TS with master...
I20260810 18:47:05.715590 3349 heartbeater.cc:511] Master 127.4.228.254:38449 requested a full tablet report, sending...
I20260810 18:47:05.718140 3069 ts_manager.cc:194] Registered new tserver with Master: f5999bdb0a8942e5a0917cf322d8c766 (127.4.228.193:43797)
I20260810 18:47:05.720570 3069 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.228.193:36871
W20260810 18:47:06.071280 3356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:47:06.071609 3356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:47:06.071777 3356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:47:06.079234 3356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:47:06.079362 3356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:47:06.079531 3356 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.194
I20260810 18:47:06.088766 3356 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:35741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.228.194
--webserver_port=0
--tserver_master_addrs=127.4.228.254:38449
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.194
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:47:06.090154 3356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:47:06.092097 3356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:47:06.106448 3362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:06.106428 3363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:06.107074 3365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:47:06.107379 3356 server_base.cc:1034] running on GCE node
I20260810 18:47:06.108099 3356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:47:06.109139 3356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:47:06.110387 3356 hybrid_clock.cc:648] HybridClock initialized: now 1786387626110338 us; error 43 us; skew 500 ppm
I20260810 18:47:06.110842 3356 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:47:06.113482 3356 webserver.cc:460] Webserver started at http://127.4.228.194:41789/ using document root <none> and password file <none>
I20260810 18:47:06.114270 3356 fs_manager.cc:359] Metadata directory not provided
I20260810 18:47:06.114414 3356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:47:06.114802 3356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:47:06.117799 3356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-1/data/instance:
uuid: "fb8dbc42d51e4a80910aaf350a47cb46"
format_stamp: "Formatted at 2026-08-10 18:47:06 on dist-test-slave-7kzw"
server_key: "9c45bb14f6bc9b7ed2ad9e5591e66f0b"
server_key_iv: "f8af94c4ce5bac58301bc535c4ca1251"
server_key_version: "encryptionkey@0"
I20260810 18:47:06.118676 3356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance:
uuid: "fb8dbc42d51e4a80910aaf350a47cb46"
format_stamp: "Formatted at 2026-08-10 18:47:06 on dist-test-slave-7kzw"
server_key: "9c45bb14f6bc9b7ed2ad9e5591e66f0b"
server_key_iv: "f8af94c4ce5bac58301bc535c4ca1251"
server_key_version: "encryptionkey@0"
I20260810 18:47:06.123988 3356 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 18:47:06.128029 3379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:06.129555 3356 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:47:06.129784 3356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-1/wal
uuid: "fb8dbc42d51e4a80910aaf350a47cb46"
format_stamp: "Formatted at 2026-08-10 18:47:06 on dist-test-slave-7kzw"
server_key: "9c45bb14f6bc9b7ed2ad9e5591e66f0b"
server_key_iv: "f8af94c4ce5bac58301bc535c4ca1251"
server_key_version: "encryptionkey@0"
I20260810 18:47:06.130024 3356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:47:06.149458 3356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:47:06.150419 3356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:47:06.150771 3356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:47:06.151787 3356 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:47:06.153355 3356 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:47:06.153496 3356 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:06.153630 3356 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:47:06.153721 3356 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:06.188766 3356 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.194:37259
I20260810 18:47:06.188862 3558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.194:37259 every 8 connection(s)
I20260810 18:47:06.190176 3356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-1/data/info.pb
I20260810 18:47:06.195568 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 3356
I20260810 18:47:06.195780 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-1/wal/instance
I20260810 18:47:06.196348 5011 external_mini_cluster.cc:1428] Setting key b66f913edc96b154f887b47fbbcc4521
I20260810 18:47:06.199781 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.228.195:0
--local_ip_for_outbound_sockets=127.4.228.195
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:38449
--builtin_ntp_servers=127.4.228.212:35741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 18:47:06.204708 3351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:06.192965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.194:60625 (local address 127.4.228.254:38449)
0810 18:47:06.193170 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:06.193175 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:06.194543 (+ 1368us) server_negotiation.cc:407] Connection header received
0810 18:47:06.197121 (+ 2578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:06.197129 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:06.197239 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:06.197357 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:06.198500 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:06.199583 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:06.201277 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:06.202031 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:06.202188 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:06.203718 (+ 1530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:06.203727 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:06.203734 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:47:06.203759 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:06.203801 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:47:06.203821 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:06.203918 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:06.204145 (+ 227us) server_negotiation.cc:299] Negotiation successful
0810 18:47:06.204262 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 18:47:06.205621 3564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:06.193437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:38449 (local address 127.4.228.194:60625)
0810 18:47:06.194350 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:06.194395 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 18:47:06.196783 (+ 2388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:47:06.197609 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:47:06.197618 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:47:06.197716 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:47:06.198287 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:06.198305 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:06.199751 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:06.199755 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:06.201118 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:06.201132 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:06.202260 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:06.202266 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:06.202617 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:06.203358 (+ 741us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:47:06.203361 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:47:06.203401 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:47:06.203482 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:47:06.203929 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:47:06.203939 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:47:06.203945 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 18:47:06.204103 (+ 158us) client_negotiation.cc:240] Negotiation successful
0810 18:47:06.204440 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":285,"threads_started":1}
I20260810 18:47:06.207500 3559 heartbeater.cc:347] Connected to a master server at 127.4.228.254:38449
I20260810 18:47:06.207854 3559 heartbeater.cc:464] Registering TS with master...
I20260810 18:47:06.208984 3559 heartbeater.cc:511] Master 127.4.228.254:38449 requested a full tablet report, sending...
I20260810 18:47:06.210868 3069 ts_manager.cc:194] Registered new tserver with Master: fb8dbc42d51e4a80910aaf350a47cb46 (127.4.228.194:37259)
I20260810 18:47:06.211915 3069 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.228.194:60625
W20260810 18:47:06.552948 3566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:47:06.553247 3566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:47:06.553416 3566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:47:06.560813 3566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:47:06.560935 3566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:47:06.561118 3566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.228.195
I20260810 18:47:06.570746 3566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:35741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.228.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.228.195
--webserver_port=0
--tserver_master_addrs=127.4.228.254:38449
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.228.195
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:47:06.572160 3566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:47:06.574067 3566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:47:06.588479 3577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:06.588603 3575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:06.588557 3573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:47:06.594786 3566 server_base.cc:1034] running on GCE node
I20260810 18:47:06.595736 3566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:47:06.596915 3566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:47:06.598244 3566 hybrid_clock.cc:648] HybridClock initialized: now 1786387626598162 us; error 43 us; skew 500 ppm
I20260810 18:47:06.598707 3566 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 18:47:06.601483 3566 webserver.cc:460] Webserver started at http://127.4.228.195:43771/ using document root <none> and password file <none>
I20260810 18:47:06.602380 3566 fs_manager.cc:359] Metadata directory not provided
I20260810 18:47:06.602569 3566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:47:06.602943 3566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:47:06.605953 3566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-2/data/instance:
uuid: "f658112c8961476f94bd049f42794ecf"
format_stamp: "Formatted at 2026-08-10 18:47:06 on dist-test-slave-7kzw"
server_key: "26baf64dc7f2ce9df8da52feca9d6eaa"
server_key_iv: "de50d4d96ad9b011e5eadc8419e8b2f8"
server_key_version: "encryptionkey@0"
I20260810 18:47:06.606858 3566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance:
uuid: "f658112c8961476f94bd049f42794ecf"
format_stamp: "Formatted at 2026-08-10 18:47:06 on dist-test-slave-7kzw"
server_key: "26baf64dc7f2ce9df8da52feca9d6eaa"
server_key_iv: "de50d4d96ad9b011e5eadc8419e8b2f8"
server_key_version: "encryptionkey@0"
I20260810 18:47:06.612222 3566 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 18:47:06.615798 3584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:06.617187 3566 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:47:06.617374 3566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-2/wal
uuid: "f658112c8961476f94bd049f42794ecf"
format_stamp: "Formatted at 2026-08-10 18:47:06 on dist-test-slave-7kzw"
server_key: "26baf64dc7f2ce9df8da52feca9d6eaa"
server_key_iv: "de50d4d96ad9b011e5eadc8419e8b2f8"
server_key_version: "encryptionkey@0"
I20260810 18:47:06.617602 3566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:47:06.648823 3566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:47:06.649780 3566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:47:06.650122 3566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:47:06.651329 3566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:47:06.652989 3566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:47:06.653081 3566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:06.653250 3566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:47:06.653347 3566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:06.688216 3566 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.195:41487
I20260810 18:47:06.688310 3757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.195:41487 every 8 connection(s)
I20260810 18:47:06.689738 3566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-2/data/info.pb
I20260810 18:47:06.695274 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 3566
I20260810 18:47:06.695490 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-2/wal/instance
I20260810 18:47:06.696061 5011 external_mini_cluster.cc:1428] Setting key 0c90dc67edd8e4b7d2f078d4e0b74480
I20260810 18:47:06.704541 3351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:06.692580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.195:56979 (local address 127.4.228.254:38449)
0810 18:47:06.692786 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:06.692791 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:06.694397 (+ 1606us) server_negotiation.cc:407] Connection header received
0810 18:47:06.697063 (+ 2666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:06.697070 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:06.697167 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:06.697283 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:06.698728 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:06.699851 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:06.701737 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:06.702279 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:06.702380 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:06.703762 (+ 1382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:06.703767 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:06.703770 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:47:06.703788 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:06.703828 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:47:06.703840 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:06.703915 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:06.704194 (+ 279us) server_negotiation.cc:299] Negotiation successful
0810 18:47:06.704289 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 18:47:06.705639 3760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:06.693168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:38449 (local address 127.4.228.195:56979)
0810 18:47:06.694180 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:06.694236 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 18:47:06.696721 (+ 2485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:47:06.697574 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:47:06.697587 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:47:06.697692 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:47:06.698525 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:06.698554 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:06.700008 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:06.700012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:06.701564 (+ 1552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:06.701578 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:06.702531 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:06.702538 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:06.702753 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:06.703446 (+ 693us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:47:06.703448 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:47:06.703490 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:47:06.703569 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:47:06.703931 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:47:06.703939 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:47:06.703946 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 18:47:06.704086 (+ 140us) client_negotiation.cc:240] Negotiation successful
0810 18:47:06.704398 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":364,"threads_started":1}
I20260810 18:47:06.707103 3758 heartbeater.cc:347] Connected to a master server at 127.4.228.254:38449
I20260810 18:47:06.707465 3758 heartbeater.cc:464] Registering TS with master...
I20260810 18:47:06.708257 3758 heartbeater.cc:511] Master 127.4.228.254:38449 requested a full tablet report, sending...
I20260810 18:47:06.709719 3069 ts_manager.cc:194] Registered new tserver with Master: f658112c8961476f94bd049f42794ecf (127.4.228.195:41487)
I20260810 18:47:06.710680 3069 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.228.195:56979
I20260810 18:47:06.712019 5011 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:47:06.722184 3351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:06.716283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35670 (local address 127.4.228.254:38449)
0810 18:47:06.716471 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:06.716475 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:06.716908 (+ 433us) server_negotiation.cc:407] Connection header received
0810 18:47:06.717042 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:06.717045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:06.717102 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:06.717183 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:06.717743 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:06.718620 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:06.719917 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:06.720362 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:06.720465 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:06.721516 (+ 1051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:06.721520 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:06.721524 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:47:06.721541 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:06.721578 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:47:06.721591 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:06.721674 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:06.721857 (+ 183us) server_negotiation.cc:299] Negotiation successful
0810 18:47:06.721930 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 18:47:06.724431 3349 heartbeater.cc:503] Master 127.4.228.254:38449 was elected leader, sending a full tablet report...
I20260810 18:47:06.727757 3067 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35670:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:47:06.736958 3067 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:47:06.768150 3783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:06.759581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:37259 (local address 127.0.0.1:49552)
0810 18:47:06.760225 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:06.760245 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:47:06.760392 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:47:06.760969 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:47:06.760973 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:47:06.760998 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:47:06.761323 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:06.761333 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:06.763068 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:06.763073 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:06.764886 (+ 1813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:06.764925 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:06.767061 (+ 2136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:06.767080 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:06.767314 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:06.767378 (+ 64us) client_negotiation.cc:769] Sending connection context
0810 18:47:06.767569 (+ 191us) client_negotiation.cc:240] Negotiation successful
0810 18:47:06.767759 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"mutex_wait_us":1,"thread_start_us":309,"threads_started":1}
I20260810 18:47:06.768275 3780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:06.753488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41190 (local address 127.4.228.193:43797)
0810 18:47:06.754226 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:06.754238 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:06.758066 (+ 3828us) server_negotiation.cc:407] Connection header received
0810 18:47:06.759368 (+ 1302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:06.759376 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:06.759605 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:06.759789 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:47:06.760762 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:06.762448 (+ 1686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:06.765470 (+ 3022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:06.766971 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:06.767165 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:06.767494 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:06.767714 (+ 220us) server_negotiation.cc:299] Negotiation successful
0810 18:47:06.767876 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":313,"threads_started":1}
I20260810 18:47:06.768443 3778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:06.752412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42478 (local address 127.4.228.195:41487)
0810 18:47:06.753366 (+ 954us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:06.753380 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:06.757979 (+ 4599us) server_negotiation.cc:407] Connection header received
0810 18:47:06.758432 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:06.758440 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:06.758787 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:06.759003 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:47:06.760165 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:06.762022 (+ 1857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:06.765411 (+ 3389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:06.766846 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:06.766985 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:06.767260 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:06.767885 (+ 625us) server_negotiation.cc:299] Negotiation successful
0810 18:47:06.768046 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":711,"thread_start_us":505,"threads_started":1}
I20260810 18:47:06.768200 3781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:06.758364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49552 (local address 127.4.228.194:37259)
0810 18:47:06.759572 (+ 1208us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:06.759581 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:06.760366 (+ 785us) server_negotiation.cc:407] Connection header received
0810 18:47:06.760574 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:06.760582 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:06.760797 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:06.760978 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:47:06.761486 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:06.762914 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:06.765113 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:06.766548 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:06.766683 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:06.766937 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:06.767632 (+ 695us) server_negotiation.cc:299] Negotiation successful
0810 18:47:06.767831 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1029,"thread_start_us":928,"threads_started":1}
I20260810 18:47:06.769284 3777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:06.752953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.193:43797 (local address 127.0.0.1:41190)
0810 18:47:06.757943 (+ 4990us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:06.757963 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:47:06.758977 (+ 1014us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:47:06.760109 (+ 1132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:47:06.760112 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:47:06.760132 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:47:06.760575 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:06.760588 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:06.762663 (+ 2075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:06.762669 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:06.765286 (+ 2617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:06.765303 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:06.767211 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:06.767218 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:06.767415 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:06.767514 (+ 99us) client_negotiation.cc:769] Sending connection context
0810 18:47:06.768625 (+ 1111us) client_negotiation.cc:240] Negotiation successful
0810 18:47:06.768809 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4807,"mutex_wait_us":93,"thread_start_us":4308,"threads_started":1}
I20260810 18:47:06.769443 3779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:06.752248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:41487 (local address 127.0.0.1:42478)
0810 18:47:06.757810 (+ 5562us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:06.757860 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 18:47:06.758160 (+ 300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:47:06.759361 (+ 1201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:47:06.759369 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:47:06.759401 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:47:06.759871 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:06.759890 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:06.762215 (+ 2325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:06.762221 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:06.765172 (+ 2951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:06.765194 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:06.767411 (+ 2217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:06.767417 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:06.767688 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:06.767719 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 18:47:06.768836 (+ 1117us) client_negotiation.cc:240] Negotiation successful
0810 18:47:06.768934 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5281,"mutex_wait_us":92,"spinlock_wait_cycles":11520,"thread_start_us":5246,"threads_started":1}
I20260810 18:47:06.773000 3457 tablet_service.cc:1467] Processing CreateTablet for tablet c16d71555a614e67aa3de03dca8353f3 (DEFAULT_TABLE table=test-table [id=12d8d27f608b42bda877576a47e11b68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:47:06.773001 3248 tablet_service.cc:1467] Processing CreateTablet for tablet c16d71555a614e67aa3de03dca8353f3 (DEFAULT_TABLE table=test-table [id=12d8d27f608b42bda877576a47e11b68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:47:06.774271 3248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c16d71555a614e67aa3de03dca8353f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:47:06.774276 3457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c16d71555a614e67aa3de03dca8353f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:47:06.777407 3658 tablet_service.cc:1467] Processing CreateTablet for tablet c16d71555a614e67aa3de03dca8353f3 (DEFAULT_TABLE table=test-table [id=12d8d27f608b42bda877576a47e11b68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:47:06.778990 3658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c16d71555a614e67aa3de03dca8353f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:47:06.786661 3784 tablet_bootstrap.cc:492] T c16d71555a614e67aa3de03dca8353f3 P fb8dbc42d51e4a80910aaf350a47cb46: Bootstrap starting.
I20260810 18:47:06.789336 3785 tablet_bootstrap.cc:492] T c16d71555a614e67aa3de03dca8353f3 P f5999bdb0a8942e5a0917cf322d8c766: Bootstrap starting.
I20260810 18:47:06.790622 3784 tablet_bootstrap.cc:654] T c16d71555a614e67aa3de03dca8353f3 P fb8dbc42d51e4a80910aaf350a47cb46: Neither blocks nor log segments found. Creating new log.
I20260810 18:47:06.791981 3784 log.cc:824] T c16d71555a614e67aa3de03dca8353f3 P fb8dbc42d51e4a80910aaf350a47cb46: Log is configured to *not* fsync() on all Append() calls
I20260810 18:47:06.793002 3786 tablet_bootstrap.cc:492] T c16d71555a614e67aa3de03dca8353f3 P f658112c8961476f94bd049f42794ecf: Bootstrap starting.
I20260810 18:47:06.793020 3785 tablet_bootstrap.cc:654] T c16d71555a614e67aa3de03dca8353f3 P f5999bdb0a8942e5a0917cf322d8c766: Neither blocks nor log segments found. Creating new log.
I20260810 18:47:06.794402 3785 log.cc:824] T c16d71555a614e67aa3de03dca8353f3 P f5999bdb0a8942e5a0917cf322d8c766: Log is configured to *not* fsync() on all Append() calls
I20260810 18:47:06.796762 3786 tablet_bootstrap.cc:654] T c16d71555a614e67aa3de03dca8353f3 P f658112c8961476f94bd049f42794ecf: Neither blocks nor log segments found. Creating new log.
I20260810 18:47:06.797221 3784 tablet_bootstrap.cc:492] T c16d71555a614e67aa3de03dca8353f3 P fb8dbc42d51e4a80910aaf350a47cb46: No bootstrap required, opened a new log
I20260810 18:47:06.797508 3784 ts_tablet_manager.cc:1397] T c16d71555a614e67aa3de03dca8353f3 P fb8dbc42d51e4a80910aaf350a47cb46: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.009s
I20260810 18:47:06.797960 3786 log.cc:824] T c16d71555a614e67aa3de03dca8353f3 P f658112c8961476f94bd049f42794ecf: Log is configured to *not* fsync() on all Append() calls
I20260810 18:47:06.799530 3785 tablet_bootstrap.cc:492] T c16d71555a614e67aa3de03dca8353f3 P f5999bdb0a8942e5a0917cf322d8c766: No bootstrap required, opened a new log
I20260810 18:47:06.799803 3785 ts_tablet_manager.cc:1397] T c16d71555a614e67aa3de03dca8353f3 P f5999bdb0a8942e5a0917cf322d8c766: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.001s
I20260810 18:47:06.800585 3786 tablet_bootstrap.cc:492] T c16d71555a614e67aa3de03dca8353f3 P f658112c8961476f94bd049f42794ecf: No bootstrap required, opened a new log
I20260810 18:47:06.800884 3786 ts_tablet_manager.cc:1397] T c16d71555a614e67aa3de03dca8353f3 P f658112c8961476f94bd049f42794ecf: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.007s
I20260810 18:47:06.808564 3784 raft_consensus.cc:348] T c16d71555a614e67aa3de03dca8353f3 P fb8dbc42d51e4a80910aaf350a47cb46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f658112c8961476f94bd049f42794ecf" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 41487 } } peers { permanent_uuid: "fb8dbc42d51e4a80910aaf350a47cb46" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 37259 } } peers { permanent_uuid: "f5999bdb0a8942e5a0917cf322d8c766" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43797 } }
I20260810 18:47:06.808806 3786 raft_consensus.cc:348] T c16d71555a614e67aa3de03dca8353f3 P f658112c8961476f94bd049f42794ecf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f658112c8961476f94bd049f42794ecf" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 41487 } } peers { permanent_uuid: "fb8dbc42d51e4a80910aaf350a47cb46" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 37259 } } peers { permanent_uuid: "f5999bdb0a8942e5a0917cf322d8c766" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43797 } }
I20260810 18:47:06.809057 3784 raft_consensus.cc:374] T c16d71555a614e67aa3de03dca8353f3 P fb8dbc42d51e4a80910aaf350a47cb46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:47:06.809150 3784 raft_consensus.cc:729] T c16d71555a614e67aa3de03dca8353f3 P fb8dbc42d51e4a80910aaf350a47cb46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb8dbc42d51e4a80910aaf350a47cb46, State: Initialized, Role: FOLLOWER
I20260810 18:47:06.809188 3786 raft_consensus.cc:374] T c16d71555a614e67aa3de03dca8353f3 P f658112c8961476f94bd049f42794ecf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:47:06.809331 3786 raft_consensus.cc:729] T c16d71555a614e67aa3de03dca8353f3 P f658112c8961476f94bd049f42794ecf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f658112c8961476f94bd049f42794ecf, State: Initialized, Role: FOLLOWER
I20260810 18:47:06.809826 3784 consensus_queue.cc:260] T c16d71555a614e67aa3de03dca8353f3 P fb8dbc42d51e4a80910aaf350a47cb46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f658112c8961476f94bd049f42794ecf" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 41487 } } peers { permanent_uuid: "fb8dbc42d51e4a80910aaf350a47cb46" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 37259 } } peers { permanent_uuid: "f5999bdb0a8942e5a0917cf322d8c766" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43797 } }
I20260810 18:47:06.809980 3786 consensus_queue.cc:260] T c16d71555a614e67aa3de03dca8353f3 P f658112c8961476f94bd049f42794ecf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f658112c8961476f94bd049f42794ecf" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 41487 } } peers { permanent_uuid: "fb8dbc42d51e4a80910aaf350a47cb46" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 37259 } } peers { permanent_uuid: "f5999bdb0a8942e5a0917cf322d8c766" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43797 } }
I20260810 18:47:06.810415 3785 raft_consensus.cc:348] T c16d71555a614e67aa3de03dca8353f3 P f5999bdb0a8942e5a0917cf322d8c766 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f658112c8961476f94bd049f42794ecf" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 41487 } } peers { permanent_uuid: "fb8dbc42d51e4a80910aaf350a47cb46" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 37259 } } peers { permanent_uuid: "f5999bdb0a8942e5a0917cf322d8c766" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43797 } }
I20260810 18:47:06.810900 3785 raft_consensus.cc:374] T c16d71555a614e67aa3de03dca8353f3 P f5999bdb0a8942e5a0917cf322d8c766 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:47:06.811012 3785 raft_consensus.cc:729] T c16d71555a614e67aa3de03dca8353f3 P f5999bdb0a8942e5a0917cf322d8c766 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5999bdb0a8942e5a0917cf322d8c766, State: Initialized, Role: FOLLOWER
I20260810 18:47:06.811364 3559 heartbeater.cc:503] Master 127.4.228.254:38449 was elected leader, sending a full tablet report...
I20260810 18:47:06.811864 3784 ts_tablet_manager.cc:1428] T c16d71555a614e67aa3de03dca8353f3 P fb8dbc42d51e4a80910aaf350a47cb46: Time spent starting tablet: real 0.014s user 0.011s sys 0.002s
I20260810 18:47:06.811954 3758 heartbeater.cc:503] Master 127.4.228.254:38449 was elected leader, sending a full tablet report...
I20260810 18:47:06.812556 3786 ts_tablet_manager.cc:1428] T c16d71555a614e67aa3de03dca8353f3 P f658112c8961476f94bd049f42794ecf: Time spent starting tablet: real 0.012s user 0.009s sys 0.001s
I20260810 18:47:06.812916 3785 consensus_queue.cc:260] T c16d71555a614e67aa3de03dca8353f3 P f5999bdb0a8942e5a0917cf322d8c766 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f658112c8961476f94bd049f42794ecf" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 41487 } } peers { permanent_uuid: "fb8dbc42d51e4a80910aaf350a47cb46" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 37259 } } peers { permanent_uuid: "f5999bdb0a8942e5a0917cf322d8c766" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43797 } }
I20260810 18:47:06.814884 3785 ts_tablet_manager.cc:1428] T c16d71555a614e67aa3de03dca8353f3 P f5999bdb0a8942e5a0917cf322d8c766: Time spent starting tablet: real 0.015s user 0.008s sys 0.007s
I20260810 18:47:06.848101 3797 raft_consensus.cc:482] T c16d71555a614e67aa3de03dca8353f3 P f5999bdb0a8942e5a0917cf322d8c766 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:47:06.848488 3797 raft_consensus.cc:504] T c16d71555a614e67aa3de03dca8353f3 P f5999bdb0a8942e5a0917cf322d8c766 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f658112c8961476f94bd049f42794ecf" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 41487 } } peers { permanent_uuid: "fb8dbc42d51e4a80910aaf350a47cb46" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 37259 } } peers { permanent_uuid: "f5999bdb0a8942e5a0917cf322d8c766" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43797 } }
I20260810 18:47:06.850286 3797 leader_election.cc:288] T c16d71555a614e67aa3de03dca8353f3 P f5999bdb0a8942e5a0917cf322d8c766 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f658112c8961476f94bd049f42794ecf (127.4.228.195:41487), fb8dbc42d51e4a80910aaf350a47cb46 (127.4.228.194:37259)
I20260810 18:47:06.858397 3798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:06.850837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.195:41487 (local address 127.4.228.193:54773)
0810 18:47:06.851489 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:06.851512 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 18:47:06.851749 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:47:06.852316 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:47:06.852319 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:47:06.852341 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:47:06.852761 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:06.852778 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:06.854290 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:06.854298 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:06.856527 (+ 2229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:06.856548 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:06.857877 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:06.857881 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:06.857989 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:06.858014 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 18:47:06.858086 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 18:47:06.858168 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":305,"threads_started":1}
I20260810 18:47:06.858563 3778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:06.850770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:54773 (local address 127.4.228.195:41487)
0810 18:47:06.850958 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:06.850963 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:06.851611 (+ 648us) server_negotiation.cc:407] Connection header received
0810 18:47:06.851960 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:06.851965 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:06.852051 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:06.852117 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:47:06.853009 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:06.854143 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:06.856728 (+ 2585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:06.857707 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:06.857859 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:06.857932 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:06.858143 (+ 211us) server_negotiation.cc:299] Negotiation successful
0810 18:47:06.858277 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 18:47:06.859454 3800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:06.851552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.194:37259 (local address 127.4.228.193:34213)
0810 18:47:06.852568 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:06.852583 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:47:06.852747 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:47:06.853151 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:47:06.853155 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:47:06.853177 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:47:06.853577 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:06.853591 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:06.855234 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:06.855240 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:06.857064 (+ 1824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:06.857075 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:06.858834 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:06.858843 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:06.859017 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:06.859053 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 18:47:06.859149 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 18:47:06.859245 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":847,"thread_start_us":367,"threads_started":1}
I20260810 18:47:06.859601 3781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:06.851950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.228.193:34213 (local address 127.4.228.194:37259)
0810 18:47:06.852150 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:06.852155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:06.852686 (+ 531us) server_negotiation.cc:407] Connection header received
0810 18:47:06.852916 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:06.852924 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:06.853021 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:06.853130 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:47:06.853836 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:06.855083 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:06.857236 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:06.858429 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:06.858624 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:06.858706 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:06.859202 (+ 496us) server_negotiation.cc:299] Negotiation successful
0810 18:47:06.859348 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 18:47:06.859623 3686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c16d71555a614e67aa3de03dca8353f3" candidate_uuid: "f5999bdb0a8942e5a0917cf322d8c766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f658112c8961476f94bd049f42794ecf" is_pre_election: true
I20260810 18:47:06.860289 3686 raft_consensus.cc:2454] T c16d71555a614e67aa3de03dca8353f3 P f658112c8961476f94bd049f42794ecf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5999bdb0a8942e5a0917cf322d8c766 in term 0.
I20260810 18:47:06.860600 3491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c16d71555a614e67aa3de03dca8353f3" candidate_uuid: "f5999bdb0a8942e5a0917cf322d8c766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb8dbc42d51e4a80910aaf350a47cb46" is_pre_election: true
I20260810 18:47:06.860946 3181 leader_election.cc:302] T c16d71555a614e67aa3de03dca8353f3 P f5999bdb0a8942e5a0917cf322d8c766 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f5999bdb0a8942e5a0917cf322d8c766, f658112c8961476f94bd049f42794ecf; no voters:
I20260810 18:47:06.861243 3491 raft_consensus.cc:2454] T c16d71555a614e67aa3de03dca8353f3 P fb8dbc42d51e4a80910aaf350a47cb46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5999bdb0a8942e5a0917cf322d8c766 in term 0.
I20260810 18:47:06.861397 3797 raft_consensus.cc:2781] T c16d71555a614e67aa3de03dca8353f3 P f5999bdb0a8942e5a0917cf322d8c766 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:47:06.861559 3797 raft_consensus.cc:482] T c16d71555a614e67aa3de03dca8353f3 P f5999bdb0a8942e5a0917cf322d8c766 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:47:06.861667 3797 raft_consensus.cc:3037] T c16d71555a614e67aa3de03dca8353f3 P f5999bdb0a8942e5a0917cf322d8c766 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:47:06.863948 3797 raft_consensus.cc:504] T c16d71555a614e67aa3de03dca8353f3 P f5999bdb0a8942e5a0917cf322d8c766 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f658112c8961476f94bd049f42794ecf" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 41487 } } peers { permanent_uuid: "fb8dbc42d51e4a80910aaf350a47cb46" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 37259 } } peers { permanent_uuid: "f5999bdb0a8942e5a0917cf322d8c766" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43797 } }
I20260810 18:47:06.864504 3797 leader_election.cc:288] T c16d71555a614e67aa3de03dca8353f3 P f5999bdb0a8942e5a0917cf322d8c766 [CANDIDATE]: Term 1 election: Requested vote from peers f658112c8961476f94bd049f42794ecf (127.4.228.195:41487), fb8dbc42d51e4a80910aaf350a47cb46 (127.4.228.194:37259)
I20260810 18:47:06.865025 3686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c16d71555a614e67aa3de03dca8353f3" candidate_uuid: "f5999bdb0a8942e5a0917cf322d8c766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f658112c8961476f94bd049f42794ecf"
I20260810 18:47:06.865150 3491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c16d71555a614e67aa3de03dca8353f3" candidate_uuid: "f5999bdb0a8942e5a0917cf322d8c766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb8dbc42d51e4a80910aaf350a47cb46"
I20260810 18:47:06.865262 3686 raft_consensus.cc:3037] T c16d71555a614e67aa3de03dca8353f3 P f658112c8961476f94bd049f42794ecf [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:47:06.865387 3491 raft_consensus.cc:3037] T c16d71555a614e67aa3de03dca8353f3 P fb8dbc42d51e4a80910aaf350a47cb46 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:47:06.867733 3686 raft_consensus.cc:2454] T c16d71555a614e67aa3de03dca8353f3 P f658112c8961476f94bd049f42794ecf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5999bdb0a8942e5a0917cf322d8c766 in term 1.
I20260810 18:47:06.867743 3491 raft_consensus.cc:2454] T c16d71555a614e67aa3de03dca8353f3 P fb8dbc42d51e4a80910aaf350a47cb46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5999bdb0a8942e5a0917cf322d8c766 in term 1.
I20260810 18:47:06.868142 3181 leader_election.cc:302] T c16d71555a614e67aa3de03dca8353f3 P f5999bdb0a8942e5a0917cf322d8c766 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f5999bdb0a8942e5a0917cf322d8c766, f658112c8961476f94bd049f42794ecf; no voters:
I20260810 18:47:06.868441 3797 raft_consensus.cc:2781] T c16d71555a614e67aa3de03dca8353f3 P f5999bdb0a8942e5a0917cf322d8c766 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:47:06.868924 3797 raft_consensus.cc:686] T c16d71555a614e67aa3de03dca8353f3 P f5999bdb0a8942e5a0917cf322d8c766 [term 1 LEADER]: Becoming Leader. State: Replica: f5999bdb0a8942e5a0917cf322d8c766, State: Running, Role: LEADER
I20260810 18:47:06.869423 3797 consensus_queue.cc:237] T c16d71555a614e67aa3de03dca8353f3 P f5999bdb0a8942e5a0917cf322d8c766 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f658112c8961476f94bd049f42794ecf" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 41487 } } peers { permanent_uuid: "fb8dbc42d51e4a80910aaf350a47cb46" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 37259 } } peers { permanent_uuid: "f5999bdb0a8942e5a0917cf322d8c766" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43797 } }
I20260810 18:47:06.874615 3069 catalog_manager.cc:5495] T c16d71555a614e67aa3de03dca8353f3 P f5999bdb0a8942e5a0917cf322d8c766 reported cstate change: term changed from 0 to 1, leader changed from <none> to f5999bdb0a8942e5a0917cf322d8c766 (127.4.228.193). New cstate: current_term: 1 leader_uuid: "f5999bdb0a8942e5a0917cf322d8c766" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f658112c8961476f94bd049f42794ecf" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 41487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb8dbc42d51e4a80910aaf350a47cb46" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 37259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5999bdb0a8942e5a0917cf322d8c766" member_type: VOTER last_known_addr { host: "127.4.228.193" port: 43797 } health_report { overall_health: HEALTHY } } }
I20260810 18:47:06.920252 3780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:06.914952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41194 (local address 127.4.228.193:43797)
0810 18:47:06.915182 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:06.915187 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:06.915208 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:47:06.915399 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:06.915403 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:06.915504 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:06.915649 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:47:06.916000 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:06.916722 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:06.918137 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:06.918623 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:06.918728 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:06.919203 (+ 475us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:47:06.919741 (+ 538us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:47:06.919860 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:06.919975 (+ 115us) server_negotiation.cc:299] Negotiation successful
0810 18:47:06.920043 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 18:47:06.928917 3686 raft_consensus.cc:1261] T c16d71555a614e67aa3de03dca8353f3 P f658112c8961476f94bd049f42794ecf [term 1 FOLLOWER]: Refusing update from remote peer f5999bdb0a8942e5a0917cf322d8c766: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:47:06.928937 3491 raft_consensus.cc:1261] T c16d71555a614e67aa3de03dca8353f3 P fb8dbc42d51e4a80910aaf350a47cb46 [term 1 FOLLOWER]: Refusing update from remote peer f5999bdb0a8942e5a0917cf322d8c766: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:47:06.929903 3801 consensus_queue.cc:1035] T c16d71555a614e67aa3de03dca8353f3 P f5999bdb0a8942e5a0917cf322d8c766 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f658112c8961476f94bd049f42794ecf" member_type: VOTER last_known_addr { host: "127.4.228.195" port: 41487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:47:06.930262 3797 consensus_queue.cc:1035] T c16d71555a614e67aa3de03dca8353f3 P f5999bdb0a8942e5a0917cf322d8c766 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb8dbc42d51e4a80910aaf350a47cb46" member_type: VOTER last_known_addr { host: "127.4.228.194" port: 37259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 18:47:06.941646 3759 tablet.cc:2369] T c16d71555a614e67aa3de03dca8353f3 P f658112c8961476f94bd049f42794ecf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:47:06.942646 3563 tablet.cc:2369] T c16d71555a614e67aa3de03dca8353f3 P fb8dbc42d51e4a80910aaf350a47cb46: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:47:06.959268 3069 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:47:06.959549 3069 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:47:06.963042 3069 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 13d3829da4f04ab19f67f4dbd3fd0d93: Sending DeleteTablet for 3 replicas of tablet c16d71555a614e67aa3de03dca8353f3
I20260810 18:47:06.964323 3248 tablet_service.cc:1514] Processing DeleteTablet for tablet c16d71555a614e67aa3de03dca8353f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:47:06 UTC) from {username='slave'} at 127.0.0.1:41190
I20260810 18:47:06.964284 3658 tablet_service.cc:1514] Processing DeleteTablet for tablet c16d71555a614e67aa3de03dca8353f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:47:06 UTC) from {username='slave'} at 127.0.0.1:42478
I20260810 18:47:06.964885 3457 tablet_service.cc:1514] Processing DeleteTablet for tablet c16d71555a614e67aa3de03dca8353f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:47:06 UTC) from {username='slave'} at 127.0.0.1:49552
I20260810 18:47:06.966785 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 3154
I20260810 18:47:06.977387 3822 tablet_replica.cc:330] T c16d71555a614e67aa3de03dca8353f3 P fb8dbc42d51e4a80910aaf350a47cb46: stopping tablet replica
I20260810 18:47:06.977377 3820 tablet_replica.cc:330] T c16d71555a614e67aa3de03dca8353f3 P f658112c8961476f94bd049f42794ecf: stopping tablet replica
I20260810 18:47:06.977867 3822 raft_consensus.cc:2229] T c16d71555a614e67aa3de03dca8353f3 P fb8dbc42d51e4a80910aaf350a47cb46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:47:06.977897 3820 raft_consensus.cc:2229] T c16d71555a614e67aa3de03dca8353f3 P f658112c8961476f94bd049f42794ecf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:47:06.978412 3822 raft_consensus.cc:2258] T c16d71555a614e67aa3de03dca8353f3 P fb8dbc42d51e4a80910aaf350a47cb46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:47:06.978510 3820 raft_consensus.cc:2258] T c16d71555a614e67aa3de03dca8353f3 P f658112c8961476f94bd049f42794ecf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:47:06.979313 3821 tablet_replica.cc:330] T c16d71555a614e67aa3de03dca8353f3 P f5999bdb0a8942e5a0917cf322d8c766: stopping tablet replica
I20260810 18:47:06.979655 3821 raft_consensus.cc:2229] T c16d71555a614e67aa3de03dca8353f3 P f5999bdb0a8942e5a0917cf322d8c766 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:47:06.980208 3821 raft_consensus.cc:2258] T c16d71555a614e67aa3de03dca8353f3 P f5999bdb0a8942e5a0917cf322d8c766 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:47:06.980876 3822 ts_tablet_manager.cc:1905] T c16d71555a614e67aa3de03dca8353f3 P fb8dbc42d51e4a80910aaf350a47cb46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:47:06.981840 3821 ts_tablet_manager.cc:1905] T c16d71555a614e67aa3de03dca8353f3 P f5999bdb0a8942e5a0917cf322d8c766: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:47:06.984155 3820 ts_tablet_manager.cc:1905] T c16d71555a614e67aa3de03dca8353f3 P f658112c8961476f94bd049f42794ecf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:47:06.988708 3822 ts_tablet_manager.cc:1918] T c16d71555a614e67aa3de03dca8353f3 P fb8dbc42d51e4a80910aaf350a47cb46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:47:06.988893 3822 log.cc:1196] T c16d71555a614e67aa3de03dca8353f3 P fb8dbc42d51e4a80910aaf350a47cb46: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-1/wal/wals/c16d71555a614e67aa3de03dca8353f3
I20260810 18:47:06.989476 3822 ts_tablet_manager.cc:1939] T c16d71555a614e67aa3de03dca8353f3 P fb8dbc42d51e4a80910aaf350a47cb46: Deleting consensus metadata
I20260810 18:47:06.989463 3821 ts_tablet_manager.cc:1918] T c16d71555a614e67aa3de03dca8353f3 P f5999bdb0a8942e5a0917cf322d8c766: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:47:06.989621 3821 log.cc:1196] T c16d71555a614e67aa3de03dca8353f3 P f5999bdb0a8942e5a0917cf322d8c766: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-0/wal/wals/c16d71555a614e67aa3de03dca8353f3
I20260810 18:47:06.990211 3821 ts_tablet_manager.cc:1939] T c16d71555a614e67aa3de03dca8353f3 P f5999bdb0a8942e5a0917cf322d8c766: Deleting consensus metadata
I20260810 18:47:06.991431 3051 catalog_manager.cc:4841] TS fb8dbc42d51e4a80910aaf350a47cb46 (127.4.228.194:37259): tablet c16d71555a614e67aa3de03dca8353f3 (table test-table [id=12d8d27f608b42bda877576a47e11b68]) successfully deleted
I20260810 18:47:06.992025 3820 ts_tablet_manager.cc:1918] T c16d71555a614e67aa3de03dca8353f3 P f658112c8961476f94bd049f42794ecf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:47:06.992025 3050 catalog_manager.cc:4841] TS f5999bdb0a8942e5a0917cf322d8c766 (127.4.228.193:43797): tablet c16d71555a614e67aa3de03dca8353f3 (table test-table [id=12d8d27f608b42bda877576a47e11b68]) successfully deleted
I20260810 18:47:06.992282 3820 log.cc:1196] T c16d71555a614e67aa3de03dca8353f3 P f658112c8961476f94bd049f42794ecf: Deleting WAL directory at /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387267874959-5011-0/minicluster-data/ts-2/wal/wals/c16d71555a614e67aa3de03dca8353f3
I20260810 18:47:06.992812 3820 ts_tablet_manager.cc:1939] T c16d71555a614e67aa3de03dca8353f3 P f658112c8961476f94bd049f42794ecf: Deleting consensus metadata
I20260810 18:47:06.994143 3051 catalog_manager.cc:4841] TS f658112c8961476f94bd049f42794ecf (127.4.228.195:41487): tablet c16d71555a614e67aa3de03dca8353f3 (table test-table [id=12d8d27f608b42bda877576a47e11b68]) successfully deleted
I20260810 18:47:06.994867 3780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:06.968603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41200 (local address 127.4.228.193:43797)
0810 18:47:06.968833 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:06.968842 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:06.985772 (+ 16930us) server_negotiation.cc:407] Connection header received
0810 18:47:06.986264 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:06.986270 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:06.986384 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:06.986496 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:06.987212 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:06.988449 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:06.990607 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:06.991245 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:06.991349 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:06.993042 (+ 1693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:06.993055 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:06.993074 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:47:06.993882 (+ 808us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:06.993944 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:47:06.993978 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:06.994060 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:06.994351 (+ 291us) server_negotiation.cc:299] Negotiation successful
0810 18:47:06.994445 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 18:47:06.995666 3340 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:47:07.091868 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 3154
I20260810 18:47:07.122915 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 3356
I20260810 18:47:07.130226 3781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:07.123874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49568 (local address 127.4.228.194:37259)
0810 18:47:07.124077 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:07.124082 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:07.124106 (+ 24us) server_negotiation.cc:407] Connection header received
0810 18:47:07.124204 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:07.124208 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:07.124307 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:07.124395 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:07.124779 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:07.125563 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:07.126865 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:07.127286 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:07.127389 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:07.128586 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:07.128596 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:07.128613 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:47:07.129287 (+ 674us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:07.129341 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:47:07.129365 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:07.129468 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:07.129655 (+ 187us) server_negotiation.cc:299] Negotiation successful
0810 18:47:07.129756 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 18:47:07.131297 3551 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:47:07.220940 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 3356
I20260810 18:47:07.249599 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 3566
I20260810 18:47:07.257160 3778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:07.250491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42480 (local address 127.4.228.195:41487)
0810 18:47:07.250730 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:07.250735 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:07.250757 (+ 22us) server_negotiation.cc:407] Connection header received
0810 18:47:07.250864 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:07.250868 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:07.250958 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:07.251066 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:07.251565 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:07.252412 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:07.253934 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:07.254335 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:07.254434 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:07.255721 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:07.255731 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:07.255777 (+ 46us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:47:07.256355 (+ 578us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:07.256408 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:47:07.256428 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:07.256525 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:07.256684 (+ 159us) server_negotiation.cc:299] Negotiation successful
0810 18:47:07.256799 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 18:47:07.258033 3746 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:47:07.346598 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 3566
I20260810 18:47:07.374433 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 3027
I20260810 18:47:07.375895 3105 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:47:07.485363 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 3027
2026-08-10T18:47:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3028 ms)
[----------] 6 tests from EncryptionPolicyTest (23809 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:47:07 dist-test-slave-7kzw krb5kdc[3832](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:47:07 dist-test-slave-7kzw krb5kdc[3832](info): set up 2 sockets
Aug 10 18:47:07 dist-test-slave-7kzw krb5kdc[3832](info): commencing operation
krb5kdc: starting...
W20260810 18:47:08.877507 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.335s user 0.007s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:47:09 dist-test-slave-7kzw krb5kdc[3832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387629, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:47:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:47:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:47:09.212535 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44227
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44227
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:47:09.580897 3850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:47:09.581221 3850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:47:09.581315 3850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:47:09.588698 3850 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:47:09.588811 3850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:47:09.588858 3850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:47:09.588927 3850 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:47:09.588972 3850 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:47:09.598801 3850 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44227
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:47:09.600099 3850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:47:09.601405 3850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:47:09.614524 3860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:09.614712 3861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:09.615378 3863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:47:09.615904 3850 server_base.cc:1034] running on GCE node
I20260810 18:47:09.616761 3850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:47:09.618263 3850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:47:09.619513 3850 hybrid_clock.cc:648] HybridClock initialized: now 1786387629619461 us; error 56 us; skew 500 ppm
I20260810 18:47:09.619896 3850 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:47:09 dist-test-slave-7kzw krb5kdc[3832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387629, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:47:09.628032 3850 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:47:09.629729 3850 webserver.cc:460] Webserver started at http://127.0.0.1:44607/ using document root <none> and password file <none>
I20260810 18:47:09.630570 3850 fs_manager.cc:359] Metadata directory not provided
I20260810 18:47:09.630739 3850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:47:09.631079 3850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:47:09.634186 3850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "cffaf7b0ba8c43a692e19450607bb007"
format_stamp: "Formatted at 2026-08-10 18:47:09 on dist-test-slave-7kzw"
server_key: "7894f42e29854d98234f4987107f9900"
server_key_iv: "5c17e0a73b8a230cf34f99850359dd9f"
server_key_version: "encryptionkey@0"
I20260810 18:47:09.635030 3850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "cffaf7b0ba8c43a692e19450607bb007"
format_stamp: "Formatted at 2026-08-10 18:47:09 on dist-test-slave-7kzw"
server_key: "7894f42e29854d98234f4987107f9900"
server_key_iv: "5c17e0a73b8a230cf34f99850359dd9f"
server_key_version: "encryptionkey@0"
I20260810 18:47:09.640486 3850 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 18:47:09.644238 3874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:09.645843 3850 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:47:09.646067 3850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "cffaf7b0ba8c43a692e19450607bb007"
format_stamp: "Formatted at 2026-08-10 18:47:09 on dist-test-slave-7kzw"
server_key: "7894f42e29854d98234f4987107f9900"
server_key_iv: "5c17e0a73b8a230cf34f99850359dd9f"
server_key_version: "encryptionkey@0"
I20260810 18:47:09.646294 3850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:47:09.686976 3850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:47:09.691308 3850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:47:09.691658 3850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:47:09.711952 3850 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44227
I20260810 18:47:09.711997 3941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44227 every 8 connection(s)
I20260810 18:47:09.713629 3850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:47:09.717855 3942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:47:09.717929 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 3850
I20260810 18:47:09.718158 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:47:09.719094 5011 external_mini_cluster.cc:1428] Setting key 52bede0403af67b2096563ad3a55b32a
I20260810 18:47:09.732786 3942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cffaf7b0ba8c43a692e19450607bb007: Bootstrap starting.
Aug 10 18:47:09 dist-test-slave-7kzw krb5kdc[3832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387629, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 18:47:09.736753 3942 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cffaf7b0ba8c43a692e19450607bb007: Neither blocks nor log segments found. Creating new log.
I20260810 18:47:09.738205 3942 log.cc:824] T 00000000000000000000000000000000 P cffaf7b0ba8c43a692e19450607bb007: Log is configured to *not* fsync() on all Append() calls
I20260810 18:47:09.741732 3942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cffaf7b0ba8c43a692e19450607bb007: No bootstrap required, opened a new log
I20260810 18:47:09.744766 3947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:09.721244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43299 (local address 127.0.0.1:44227)
0810 18:47:09.722105 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:09.722117 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:09.722168 (+ 51us) server_negotiation.cc:407] Connection header received
0810 18:47:09.724467 (+ 2299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:09.724491 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:09.724894 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:09.725230 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:09.727008 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:09.728317 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:09.730237 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:09.731076 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:09.731238 (+ 162us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:09.736283 (+ 5045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:09.736296 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:09.736316 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:47:09.736363 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:09.740081 (+ 3718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:09.741089 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:09.741095 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:09.741102 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:09.741283 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:09.741904 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:09.741908 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:09.741910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:09.742555 (+ 645us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:09.742868 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:09.743345 (+ 477us) server_negotiation.cc:299] Negotiation successful
0810 18:47:09.743661 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":426,"threads_started":1}
I20260810 18:47:09.749195 5011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:47:09.751353 3942 raft_consensus.cc:348] T 00000000000000000000000000000000 P cffaf7b0ba8c43a692e19450607bb007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cffaf7b0ba8c43a692e19450607bb007" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44227 } }
I20260810 18:47:09.751863 3942 raft_consensus.cc:374] T 00000000000000000000000000000000 P cffaf7b0ba8c43a692e19450607bb007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:47:09.751999 3942 raft_consensus.cc:729] T 00000000000000000000000000000000 P cffaf7b0ba8c43a692e19450607bb007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cffaf7b0ba8c43a692e19450607bb007, State: Initialized, Role: FOLLOWER
I20260810 18:47:09.752772 3942 consensus_queue.cc:260] T 00000000000000000000000000000000 P cffaf7b0ba8c43a692e19450607bb007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cffaf7b0ba8c43a692e19450607bb007" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44227 } }
I20260810 18:47:09.753126 3942 raft_consensus.cc:388] T 00000000000000000000000000000000 P cffaf7b0ba8c43a692e19450607bb007 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:47:09.753289 3942 raft_consensus.cc:482] T 00000000000000000000000000000000 P cffaf7b0ba8c43a692e19450607bb007 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:47:09.753506 3942 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cffaf7b0ba8c43a692e19450607bb007 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:47:09.756954 3942 raft_consensus.cc:504] T 00000000000000000000000000000000 P cffaf7b0ba8c43a692e19450607bb007 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cffaf7b0ba8c43a692e19450607bb007" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44227 } }
I20260810 18:47:09.757623 3942 leader_election.cc:302] T 00000000000000000000000000000000 P cffaf7b0ba8c43a692e19450607bb007 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cffaf7b0ba8c43a692e19450607bb007; no voters:
I20260810 18:47:09.758360 3942 leader_election.cc:288] T 00000000000000000000000000000000 P cffaf7b0ba8c43a692e19450607bb007 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:47:09.758457 3957 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cffaf7b0ba8c43a692e19450607bb007 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:47:09.760010 3957 raft_consensus.cc:686] T 00000000000000000000000000000000 P cffaf7b0ba8c43a692e19450607bb007 [term 1 LEADER]: Becoming Leader. State: Replica: cffaf7b0ba8c43a692e19450607bb007, State: Running, Role: LEADER
I20260810 18:47:09.760703 3957 consensus_queue.cc:237] T 00000000000000000000000000000000 P cffaf7b0ba8c43a692e19450607bb007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cffaf7b0ba8c43a692e19450607bb007" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44227 } }
I20260810 18:47:09.761018 3942 sys_catalog.cc:564] T 00000000000000000000000000000000 P cffaf7b0ba8c43a692e19450607bb007 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:47:09.763603 3963 sys_catalog.cc:455] T 00000000000000000000000000000000 P cffaf7b0ba8c43a692e19450607bb007 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cffaf7b0ba8c43a692e19450607bb007. Latest consensus state: current_term: 1 leader_uuid: "cffaf7b0ba8c43a692e19450607bb007" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cffaf7b0ba8c43a692e19450607bb007" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44227 } } }
I20260810 18:47:09.764102 3963 sys_catalog.cc:458] T 00000000000000000000000000000000 P cffaf7b0ba8c43a692e19450607bb007 [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:09.764804 3958 sys_catalog.cc:455] T 00000000000000000000000000000000 P cffaf7b0ba8c43a692e19450607bb007 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cffaf7b0ba8c43a692e19450607bb007" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cffaf7b0ba8c43a692e19450607bb007" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44227 } } }
I20260810 18:47:09.765194 3958 sys_catalog.cc:458] T 00000000000000000000000000000000 P cffaf7b0ba8c43a692e19450607bb007 [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:09.768721 3966 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:47:09.774816 3966 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:47:09.779301 3947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:09.755596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55751 (local address 127.0.0.1:44227)
0810 18:47:09.755801 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:09.755809 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:09.756395 (+ 586us) server_negotiation.cc:407] Connection header received
0810 18:47:09.756544 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:09.756549 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:09.756621 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:09.756718 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:09.758843 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:09.760384 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:09.762269 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:09.763316 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:09.763477 (+ 161us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:09.767926 (+ 4449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:09.767937 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:09.767949 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:47:09.768009 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:09.773278 (+ 5269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:09.774631 (+ 1353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:09.774640 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:09.774646 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:09.774878 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:09.775652 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:09.775659 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:09.775669 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:09.776302 (+ 633us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:09.776678 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:09.778445 (+ 1767us) server_negotiation.cc:299] Negotiation successful
0810 18:47:09.778750 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 18:47:09.790275 3966 catalog_manager.cc:1347] Generated new cluster ID: 7b5431f8e7a34b22949a91692d1f720f
I20260810 18:47:09.790421 3966 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:47:09.808492 3966 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:47:09.809816 3966 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:47:09.823824 3966 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cffaf7b0ba8c43a692e19450607bb007: Generated new TSK 0
I20260810 18:47:09.824931 3966 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:47:09.837659 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 3850
I20260810 18:47:09.839174 3936 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:47:09.933871 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 3850
W20260810 18:47:09.962415 3829 connection.cc:535] client connection to 127.0.0.1:44227 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44227): Connection reset by peer (error 104)
2026-08-10T18:47:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2440 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:47:10 dist-test-slave-7kzw krb5kdc[3989](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:47:10 dist-test-slave-7kzw krb5kdc[3989](info): set up 2 sockets
Aug 10 18:47:10 dist-test-slave-7kzw krb5kdc[3989](info): commencing operation
krb5kdc: starting...
W20260810 18:47:11.274420 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.293s user 0.006s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:47:11 dist-test-slave-7kzw krb5kdc[3989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:47:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:47:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:47:11.605979 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34421
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34421
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:47:11.975653 4011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:47:11.975967 4011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:47:11.976138 4011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:47:11.983584 4011 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:47:11.983697 4011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:47:11.983783 4011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:47:11.983892 4011 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:47:11.983958 4011 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:47:11.994064 4011 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34421
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:47:11.995426 4011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:47:11.996825 4011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:47:12.009330 4021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:12.009343 4020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:12.009658 4023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:47:12.010200 4011 server_base.cc:1034] running on GCE node
I20260810 18:47:12.011066 4011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:47:12.012517 4011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:47:12.013792 4011 hybrid_clock.cc:648] HybridClock initialized: now 1786387632013723 us; error 48 us; skew 500 ppm
I20260810 18:47:12.014132 4011 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:47:12 dist-test-slave-7kzw krb5kdc[3989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387632, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:47:12.017737 4011 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:47:12.019284 4011 webserver.cc:460] Webserver started at http://127.0.0.1:40443/ using document root <none> and password file <none>
I20260810 18:47:12.020038 4011 fs_manager.cc:359] Metadata directory not provided
I20260810 18:47:12.020195 4011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:47:12.020529 4011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:47:12.023629 4011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "daa21535d6cf4cffb9deabd4a8e7a928"
format_stamp: "Formatted at 2026-08-10 18:47:12 on dist-test-slave-7kzw"
server_key: "4ea3f5dacdd6a0b1975bc7bcb35f70bb"
server_key_iv: "781924eeec52e7f6e9d9255708e0e9dc"
server_key_version: "encryptionkey@0"
I20260810 18:47:12.024425 4011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "daa21535d6cf4cffb9deabd4a8e7a928"
format_stamp: "Formatted at 2026-08-10 18:47:12 on dist-test-slave-7kzw"
server_key: "4ea3f5dacdd6a0b1975bc7bcb35f70bb"
server_key_iv: "781924eeec52e7f6e9d9255708e0e9dc"
server_key_version: "encryptionkey@0"
I20260810 18:47:12.029976 4011 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 18:47:12.033954 4035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:12.035501 4011 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:47:12.035696 4011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "daa21535d6cf4cffb9deabd4a8e7a928"
format_stamp: "Formatted at 2026-08-10 18:47:12 on dist-test-slave-7kzw"
server_key: "4ea3f5dacdd6a0b1975bc7bcb35f70bb"
server_key_iv: "781924eeec52e7f6e9d9255708e0e9dc"
server_key_version: "encryptionkey@0"
I20260810 18:47:12.035912 4011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:47:12.055586 4011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:47:12.059855 4011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:47:12.060154 4011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:47:12.079862 4011 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34421
I20260810 18:47:12.079911 4103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34421 every 8 connection(s)
I20260810 18:47:12.081303 4011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:47:12.085157 4104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:47:12.091416 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 4011
I20260810 18:47:12.091609 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:47:12.092188 5011 external_mini_cluster.cc:1428] Setting key 6489dff0e7fc8a9bbd71ed9699755a91
I20260810 18:47:12.096498 4104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P daa21535d6cf4cffb9deabd4a8e7a928: Bootstrap starting.
I20260810 18:47:12.099972 4104 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P daa21535d6cf4cffb9deabd4a8e7a928: Neither blocks nor log segments found. Creating new log.
I20260810 18:47:12.101284 4104 log.cc:824] T 00000000000000000000000000000000 P daa21535d6cf4cffb9deabd4a8e7a928: Log is configured to *not* fsync() on all Append() calls
I20260810 18:47:12.104777 4104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P daa21535d6cf4cffb9deabd4a8e7a928: No bootstrap required, opened a new log
I20260810 18:47:12.107422 4109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:12.094253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35789 (local address 127.0.0.1:34421)
0810 18:47:12.095367 (+ 1114us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:12.095380 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:12.095427 (+ 47us) server_negotiation.cc:407] Connection header received
0810 18:47:12.097425 (+ 1998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:12.097451 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:12.097885 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:12.098253 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:12.098911 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:12.100434 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:12.102354 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:12.103214 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:12.103361 (+ 147us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:12.104819 (+ 1458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:12.104831 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:12.104851 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:47:12.105441 (+ 590us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:12.105521 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:47:12.105546 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:12.105671 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:12.106071 (+ 400us) server_negotiation.cc:299] Negotiation successful
0810 18:47:12.106261 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":873,"thread_start_us":653,"threads_started":1}
I20260810 18:47:12.112007 5011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:47:12.114370 4104 raft_consensus.cc:348] T 00000000000000000000000000000000 P daa21535d6cf4cffb9deabd4a8e7a928 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daa21535d6cf4cffb9deabd4a8e7a928" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34421 } }
I20260810 18:47:12.114972 4104 raft_consensus.cc:374] T 00000000000000000000000000000000 P daa21535d6cf4cffb9deabd4a8e7a928 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:47:12.115182 4104 raft_consensus.cc:729] T 00000000000000000000000000000000 P daa21535d6cf4cffb9deabd4a8e7a928 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: daa21535d6cf4cffb9deabd4a8e7a928, State: Initialized, Role: FOLLOWER
I20260810 18:47:12.115935 4104 consensus_queue.cc:260] T 00000000000000000000000000000000 P daa21535d6cf4cffb9deabd4a8e7a928 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daa21535d6cf4cffb9deabd4a8e7a928" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34421 } }
I20260810 18:47:12.116252 4104 raft_consensus.cc:388] T 00000000000000000000000000000000 P daa21535d6cf4cffb9deabd4a8e7a928 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:47:12.116441 4104 raft_consensus.cc:482] T 00000000000000000000000000000000 P daa21535d6cf4cffb9deabd4a8e7a928 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:47:12.116709 4104 raft_consensus.cc:3037] T 00000000000000000000000000000000 P daa21535d6cf4cffb9deabd4a8e7a928 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:47:12.119540 4104 raft_consensus.cc:504] T 00000000000000000000000000000000 P daa21535d6cf4cffb9deabd4a8e7a928 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daa21535d6cf4cffb9deabd4a8e7a928" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34421 } }
I20260810 18:47:12.120141 4104 leader_election.cc:302] T 00000000000000000000000000000000 P daa21535d6cf4cffb9deabd4a8e7a928 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: daa21535d6cf4cffb9deabd4a8e7a928; no voters:
I20260810 18:47:12.120970 4104 leader_election.cc:288] T 00000000000000000000000000000000 P daa21535d6cf4cffb9deabd4a8e7a928 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:47:12.121094 4122 raft_consensus.cc:2781] T 00000000000000000000000000000000 P daa21535d6cf4cffb9deabd4a8e7a928 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:47:12.121899 4122 raft_consensus.cc:686] T 00000000000000000000000000000000 P daa21535d6cf4cffb9deabd4a8e7a928 [term 1 LEADER]: Becoming Leader. State: Replica: daa21535d6cf4cffb9deabd4a8e7a928, State: Running, Role: LEADER
I20260810 18:47:12.123423 4109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:12.116651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34395 (local address 127.0.0.1:34421)
0810 18:47:12.116923 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:12.116932 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:12.117280 (+ 348us) server_negotiation.cc:407] Connection header received
0810 18:47:12.117510 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:12.117515 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:12.117593 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:12.117738 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:12.118303 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:12.119140 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:12.120579 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:12.121127 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:12.121254 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:12.122594 (+ 1340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:12.122600 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:12.122605 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:47:12.122632 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:12.122716 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:47:12.122732 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:12.122864 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:12.123090 (+ 226us) server_negotiation.cc:299] Negotiation successful
0810 18:47:12.123145 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260810 18:47:12.122876 4122 consensus_queue.cc:237] T 00000000000000000000000000000000 P daa21535d6cf4cffb9deabd4a8e7a928 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daa21535d6cf4cffb9deabd4a8e7a928" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34421 } }
I20260810 18:47:12.125469 4104 sys_catalog.cc:564] T 00000000000000000000000000000000 P daa21535d6cf4cffb9deabd4a8e7a928 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:47:12.126884 4123 sys_catalog.cc:455] T 00000000000000000000000000000000 P daa21535d6cf4cffb9deabd4a8e7a928 [sys.catalog]: SysCatalogTable state changed. Reason: New leader daa21535d6cf4cffb9deabd4a8e7a928. Latest consensus state: current_term: 1 leader_uuid: "daa21535d6cf4cffb9deabd4a8e7a928" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daa21535d6cf4cffb9deabd4a8e7a928" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34421 } } }
I20260810 18:47:12.126974 4124 sys_catalog.cc:455] T 00000000000000000000000000000000 P daa21535d6cf4cffb9deabd4a8e7a928 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "daa21535d6cf4cffb9deabd4a8e7a928" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daa21535d6cf4cffb9deabd4a8e7a928" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34421 } } }
I20260810 18:47:12.127595 4124 sys_catalog.cc:458] T 00000000000000000000000000000000 P daa21535d6cf4cffb9deabd4a8e7a928 [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:12.127351 4123 sys_catalog.cc:458] T 00000000000000000000000000000000 P daa21535d6cf4cffb9deabd4a8e7a928 [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:12.129187 4134 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:47:12.134872 4134 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:47:12.145048 4134 catalog_manager.cc:1347] Generated new cluster ID: 02d7f006410f403cae10ad99a01cebef
I20260810 18:47:12.145197 4134 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:47:12.159067 4134 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:47:12.160317 4134 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:47:12.175510 4134 catalog_manager.cc:5868] T 00000000000000000000000000000000 P daa21535d6cf4cffb9deabd4a8e7a928: Generated new TSK 0
I20260810 18:47:12.176391 4134 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:47:12.189767 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 4011
W20260810 18:47:12.190929 4096 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35789
W20260810 18:47:12.191798 5011 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 4011: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:47:12.191993 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 4011
W20260810 18:47:12.223227 3987 connection.cc:535] client connection to 127.0.0.1:34421 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34421): Connection reset by peer (error 104)
2026-08-10T18:47:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2260 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:47:12 dist-test-slave-7kzw krb5kdc[4155](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:47:12 dist-test-slave-7kzw krb5kdc[4155](info): set up 2 sockets
Aug 10 18:47:12 dist-test-slave-7kzw krb5kdc[4155](info): commencing operation
krb5kdc: starting...
W20260810 18:47:13.528213 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.286s user 0.004s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:47:13 dist-test-slave-7kzw krb5kdc[4155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:47:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:47:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:47:13.877851 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44239
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44239
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:47:14.235044 4169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:47:14.235414 4169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:47:14.235601 4169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:47:14.242769 4169 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:47:14.242914 4169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:47:14.243027 4169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:47:14.243146 4169 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:47:14.243227 4169 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:47:14.253244 4169 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44239
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:47:14.254731 4169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:47:14.256105 4169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:47:14.268689 4176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:14.268685 4177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:14.269117 4180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:47:14.269665 4169 server_base.cc:1034] running on GCE node
I20260810 18:47:14.270522 4169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:47:14.272027 4169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:47:14.273252 4169 hybrid_clock.cc:648] HybridClock initialized: now 1786387634273210 us; error 26 us; skew 500 ppm
I20260810 18:47:14.273623 4169 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:47:14 dist-test-slave-7kzw krb5kdc[4155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387634, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:47:14.276954 4169 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:47:14.278542 4169 webserver.cc:460] Webserver started at http://127.0.0.1:36923/ using document root <none> and password file <none>
I20260810 18:47:14.279258 4169 fs_manager.cc:359] Metadata directory not provided
I20260810 18:47:14.279394 4169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:47:14.279735 4169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:47:14.282855 4169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "ace66baa603f42d3a16f79a2d61f1163"
format_stamp: "Formatted at 2026-08-10 18:47:14 on dist-test-slave-7kzw"
server_key: "86d18ee5a4a73bac6ab83080596cc31b"
server_key_iv: "b15d145fd588dd06eb817341675656b2"
server_key_version: "encryptionkey@0"
I20260810 18:47:14.283655 4169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "ace66baa603f42d3a16f79a2d61f1163"
format_stamp: "Formatted at 2026-08-10 18:47:14 on dist-test-slave-7kzw"
server_key: "86d18ee5a4a73bac6ab83080596cc31b"
server_key_iv: "b15d145fd588dd06eb817341675656b2"
server_key_version: "encryptionkey@0"
I20260810 18:47:14.289702 4169 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260810 18:47:14.293303 4191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:14.294723 4169 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:47:14.294917 4169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "ace66baa603f42d3a16f79a2d61f1163"
format_stamp: "Formatted at 2026-08-10 18:47:14 on dist-test-slave-7kzw"
server_key: "86d18ee5a4a73bac6ab83080596cc31b"
server_key_iv: "b15d145fd588dd06eb817341675656b2"
server_key_version: "encryptionkey@0"
I20260810 18:47:14.295122 4169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:47:14.301035 4169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:47:14.305193 4169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:47:14.305509 4169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:47:14.324641 4169 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44239
I20260810 18:47:14.324667 4265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44239 every 8 connection(s)
I20260810 18:47:14.326206 4169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:47:14.330087 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 4169
I20260810 18:47:14.330296 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:47:14.330669 4266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:47:14.330866 5011 external_mini_cluster.cc:1428] Setting key acfba4cf8e8d118640921aaa7346e931
I20260810 18:47:14.339753 4272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:14.332512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32971 (local address 127.0.0.1:44239)
0810 18:47:14.333349 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:14.333361 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:14.333877 (+ 516us) server_negotiation.cc:407] Connection header received
0810 18:47:14.336190 (+ 2313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:14.336212 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:14.336558 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:14.337017 (+ 459us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:14.338106 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:14.338112 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:14.338127 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:47:14.338142 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:14.338208 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:47:14.338227 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:14.338515 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:14.338686 (+ 171us) server_negotiation.cc:299] Negotiation successful
0810 18:47:14.338863 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":375,"threads_started":1}
I20260810 18:47:14.344026 5011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:47:14.346742 4266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ace66baa603f42d3a16f79a2d61f1163: Bootstrap starting.
I20260810 18:47:14.351469 4266 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ace66baa603f42d3a16f79a2d61f1163: Neither blocks nor log segments found. Creating new log.
I20260810 18:47:14.352754 4272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:14.350120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59107 (local address 127.0.0.1:44239)
0810 18:47:14.350326 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:14.350333 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:14.350673 (+ 340us) server_negotiation.cc:407] Connection header received
0810 18:47:14.350818 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:14.350824 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:14.350894 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:14.351021 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:14.352035 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:14.352043 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:14.352051 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:47:14.352074 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:14.352125 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:47:14.352148 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:14.352277 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:14.352353 (+ 76us) server_negotiation.cc:299] Negotiation successful
0810 18:47:14.352412 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 18:47:14.353073 4266 log.cc:824] T 00000000000000000000000000000000 P ace66baa603f42d3a16f79a2d61f1163: Log is configured to *not* fsync() on all Append() calls
I20260810 18:47:14.356356 4266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ace66baa603f42d3a16f79a2d61f1163: No bootstrap required, opened a new log
I20260810 18:47:14.365024 4266 raft_consensus.cc:348] T 00000000000000000000000000000000 P ace66baa603f42d3a16f79a2d61f1163 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ace66baa603f42d3a16f79a2d61f1163" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44239 } }
I20260810 18:47:14.365484 4266 raft_consensus.cc:374] T 00000000000000000000000000000000 P ace66baa603f42d3a16f79a2d61f1163 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:47:14.365612 4266 raft_consensus.cc:729] T 00000000000000000000000000000000 P ace66baa603f42d3a16f79a2d61f1163 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ace66baa603f42d3a16f79a2d61f1163, State: Initialized, Role: FOLLOWER
I20260810 18:47:14.366511 4266 consensus_queue.cc:260] T 00000000000000000000000000000000 P ace66baa603f42d3a16f79a2d61f1163 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ace66baa603f42d3a16f79a2d61f1163" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44239 } }
I20260810 18:47:14.366804 4266 raft_consensus.cc:388] T 00000000000000000000000000000000 P ace66baa603f42d3a16f79a2d61f1163 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:47:14.366973 4266 raft_consensus.cc:482] T 00000000000000000000000000000000 P ace66baa603f42d3a16f79a2d61f1163 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:47:14.367179 4266 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ace66baa603f42d3a16f79a2d61f1163 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:47:14.369807 4266 raft_consensus.cc:504] T 00000000000000000000000000000000 P ace66baa603f42d3a16f79a2d61f1163 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ace66baa603f42d3a16f79a2d61f1163" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44239 } }
I20260810 18:47:14.370563 4266 leader_election.cc:302] T 00000000000000000000000000000000 P ace66baa603f42d3a16f79a2d61f1163 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ace66baa603f42d3a16f79a2d61f1163; no voters:
I20260810 18:47:14.371309 4266 leader_election.cc:288] T 00000000000000000000000000000000 P ace66baa603f42d3a16f79a2d61f1163 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:47:14.371508 4283 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ace66baa603f42d3a16f79a2d61f1163 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:47:14.372238 4283 raft_consensus.cc:686] T 00000000000000000000000000000000 P ace66baa603f42d3a16f79a2d61f1163 [term 1 LEADER]: Becoming Leader. State: Replica: ace66baa603f42d3a16f79a2d61f1163, State: Running, Role: LEADER
I20260810 18:47:14.372790 4283 consensus_queue.cc:237] T 00000000000000000000000000000000 P ace66baa603f42d3a16f79a2d61f1163 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ace66baa603f42d3a16f79a2d61f1163" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44239 } }
I20260810 18:47:14.373677 4266 sys_catalog.cc:564] T 00000000000000000000000000000000 P ace66baa603f42d3a16f79a2d61f1163 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:47:14.376096 4285 sys_catalog.cc:455] T 00000000000000000000000000000000 P ace66baa603f42d3a16f79a2d61f1163 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ace66baa603f42d3a16f79a2d61f1163. Latest consensus state: current_term: 1 leader_uuid: "ace66baa603f42d3a16f79a2d61f1163" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ace66baa603f42d3a16f79a2d61f1163" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44239 } } }
I20260810 18:47:14.376506 4285 sys_catalog.cc:458] T 00000000000000000000000000000000 P ace66baa603f42d3a16f79a2d61f1163 [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:14.377045 4284 sys_catalog.cc:455] T 00000000000000000000000000000000 P ace66baa603f42d3a16f79a2d61f1163 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ace66baa603f42d3a16f79a2d61f1163" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ace66baa603f42d3a16f79a2d61f1163" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44239 } } }
I20260810 18:47:14.377355 4284 sys_catalog.cc:458] T 00000000000000000000000000000000 P ace66baa603f42d3a16f79a2d61f1163 [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:14.379274 4291 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:47:14.384596 4291 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:47:14.394255 4291 catalog_manager.cc:1347] Generated new cluster ID: 2f59d5a4ee604b2e92ee20b9e07f428a
I20260810 18:47:14.394400 4291 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:47:14.435273 4291 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:47:14.435783 4291 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:47:14.443348 4291 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ace66baa603f42d3a16f79a2d61f1163: Generated new TSK 0
I20260810 18:47:14.444239 4291 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:47:14.449321 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 4169
W20260810 18:47:14.450569 4258 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:32971
W20260810 18:47:14.451333 5011 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 4169: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:47:14.451522 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 4169
2026-08-10T18:47:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2256 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:47:14 dist-test-slave-7kzw krb5kdc[4309](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:47:14 dist-test-slave-7kzw krb5kdc[4309](info): set up 2 sockets
Aug 10 18:47:14 dist-test-slave-7kzw krb5kdc[4309](info): commencing operation
krb5kdc: starting...
W20260810 18:47:15.816174 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.317s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:47:16 dist-test-slave-7kzw krb5kdc[4309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:47:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:47:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:47:16.150377 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35035
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35035
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:47:16.509158 4330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:47:16.509554 4330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:47:16.509740 4330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:47:16.517061 4330 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:47:16.517227 4330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:47:16.517349 4330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:47:16.517467 4330 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:47:16.517560 4330 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:47:16.527890 4330 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:47:16.529392 4330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:47:16.530920 4330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:47:16.543731 4342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:16.543823 4340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:16.543820 4345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:47:16.544483 4330 server_base.cc:1034] running on GCE node
I20260810 18:47:16.545364 4330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:47:16.546907 4330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:47:16.548177 4330 hybrid_clock.cc:648] HybridClock initialized: now 1786387636548108 us; error 57 us; skew 500 ppm
I20260810 18:47:16.548560 4330 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:47:16 dist-test-slave-7kzw krb5kdc[4309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387636, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:47:16.555361 4330 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:47:16.556963 4330 webserver.cc:460] Webserver started at http://127.0.0.1:36407/ using document root <none> and password file <none>
I20260810 18:47:16.557729 4330 fs_manager.cc:359] Metadata directory not provided
I20260810 18:47:16.557883 4330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:47:16.558238 4330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:47:16.561398 4330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "4f8fd8f4a66f4005a91564f0b537a9b4"
format_stamp: "Formatted at 2026-08-10 18:47:16 on dist-test-slave-7kzw"
server_key: "6e8baa3862b31a7b2a4bd7cf33ad7768"
server_key_iv: "a2cc5c90493c56e113a73eeed1bf4890"
server_key_version: "encryptionkey@0"
I20260810 18:47:16.562278 4330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "4f8fd8f4a66f4005a91564f0b537a9b4"
format_stamp: "Formatted at 2026-08-10 18:47:16 on dist-test-slave-7kzw"
server_key: "6e8baa3862b31a7b2a4bd7cf33ad7768"
server_key_iv: "a2cc5c90493c56e113a73eeed1bf4890"
server_key_version: "encryptionkey@0"
I20260810 18:47:16.567885 4330 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 18:47:16.571707 4355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:16.573105 4330 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:47:16.573333 4330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "4f8fd8f4a66f4005a91564f0b537a9b4"
format_stamp: "Formatted at 2026-08-10 18:47:16 on dist-test-slave-7kzw"
server_key: "6e8baa3862b31a7b2a4bd7cf33ad7768"
server_key_iv: "a2cc5c90493c56e113a73eeed1bf4890"
server_key_version: "encryptionkey@0"
I20260810 18:47:16.573549 4330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:47:16.598266 4330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:47:16.602910 4330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:47:16.603173 4330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:47:16.622304 4330 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35035
I20260810 18:47:16.622337 4434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35035 every 8 connection(s)
I20260810 18:47:16.623843 4330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:47:16.628233 4435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:47:16.633193 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 4330
I20260810 18:47:16.633414 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:47:16.633951 5011 external_mini_cluster.cc:1428] Setting key 44a18012489930510061fde519875d42
I20260810 18:47:16.642864 4435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f8fd8f4a66f4005a91564f0b537a9b4: Bootstrap starting.
I20260810 18:47:16.647122 4435 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f8fd8f4a66f4005a91564f0b537a9b4: Neither blocks nor log segments found. Creating new log.
I20260810 18:47:16.648615 4435 log.cc:824] T 00000000000000000000000000000000 P 4f8fd8f4a66f4005a91564f0b537a9b4: Log is configured to *not* fsync() on all Append() calls
Aug 10 18:47:16 dist-test-slave-7kzw krb5kdc[4309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 18:47:16.652395 4435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f8fd8f4a66f4005a91564f0b537a9b4: No bootstrap required, opened a new log
I20260810 18:47:16.664211 4440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:16.635913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57133 (local address 127.0.0.1:35035)
0810 18:47:16.638114 (+ 2201us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:16.638136 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:16.638201 (+ 65us) server_negotiation.cc:407] Connection header received
0810 18:47:16.640622 (+ 2421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:16.640655 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:16.641050 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:16.641380 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:16.643589 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:16.644963 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:16.646989 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:16.647795 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:16.647948 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:16.653241 (+ 5293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:16.653259 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:16.653285 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:47:16.653338 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:16.658176 (+ 4838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:16.659178 (+ 1002us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:16.659190 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:16.659201 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:16.659468 (+ 267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:16.659963 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:16.659970 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:16.659974 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:16.660904 (+ 930us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:16.661330 (+ 426us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:16.661866 (+ 536us) server_negotiation.cc:299] Negotiation successful
0810 18:47:16.662443 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1909,"thread_start_us":1632,"threads_started":1}
I20260810 18:47:16.665238 4435 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f8fd8f4a66f4005a91564f0b537a9b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f8fd8f4a66f4005a91564f0b537a9b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35035 } }
I20260810 18:47:16.665678 4435 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f8fd8f4a66f4005a91564f0b537a9b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:47:16.665829 4435 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f8fd8f4a66f4005a91564f0b537a9b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f8fd8f4a66f4005a91564f0b537a9b4, State: Initialized, Role: FOLLOWER
I20260810 18:47:16.666709 4435 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f8fd8f4a66f4005a91564f0b537a9b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f8fd8f4a66f4005a91564f0b537a9b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35035 } }
I20260810 18:47:16.667047 4435 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f8fd8f4a66f4005a91564f0b537a9b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:47:16.667225 4435 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f8fd8f4a66f4005a91564f0b537a9b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:47:16.667490 4435 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f8fd8f4a66f4005a91564f0b537a9b4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:47:16.669060 5011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:47:16.670455 4435 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f8fd8f4a66f4005a91564f0b537a9b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f8fd8f4a66f4005a91564f0b537a9b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35035 } }
I20260810 18:47:16.671116 4435 leader_election.cc:302] T 00000000000000000000000000000000 P 4f8fd8f4a66f4005a91564f0b537a9b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f8fd8f4a66f4005a91564f0b537a9b4; no voters:
I20260810 18:47:16.671886 4435 leader_election.cc:288] T 00000000000000000000000000000000 P 4f8fd8f4a66f4005a91564f0b537a9b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:47:16.671999 4443 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f8fd8f4a66f4005a91564f0b537a9b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:47:16.673524 4443 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f8fd8f4a66f4005a91564f0b537a9b4 [term 1 LEADER]: Becoming Leader. State: Replica: 4f8fd8f4a66f4005a91564f0b537a9b4, State: Running, Role: LEADER
I20260810 18:47:16.674595 4435 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f8fd8f4a66f4005a91564f0b537a9b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:47:16.674386 4443 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f8fd8f4a66f4005a91564f0b537a9b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f8fd8f4a66f4005a91564f0b537a9b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35035 } }
I20260810 18:47:16.681597 4449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f8fd8f4a66f4005a91564f0b537a9b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f8fd8f4a66f4005a91564f0b537a9b4. Latest consensus state: current_term: 1 leader_uuid: "4f8fd8f4a66f4005a91564f0b537a9b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f8fd8f4a66f4005a91564f0b537a9b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35035 } } }
I20260810 18:47:16.681562 4448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f8fd8f4a66f4005a91564f0b537a9b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f8fd8f4a66f4005a91564f0b537a9b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f8fd8f4a66f4005a91564f0b537a9b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35035 } } }
I20260810 18:47:16.682147 4449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f8fd8f4a66f4005a91564f0b537a9b4 [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:16.682147 4448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f8fd8f4a66f4005a91564f0b537a9b4 [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:16.684445 4465 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:47:16.691995 4465 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:47:16.706095 4465 catalog_manager.cc:1347] Generated new cluster ID: 255b0db494274f6eab703f4cdebd026f
I20260810 18:47:16.706269 4465 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:47:16.707329 4440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:16.683265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58269 (local address 127.0.0.1:35035)
0810 18:47:16.683517 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:16.683527 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:16.687071 (+ 3544us) server_negotiation.cc:407] Connection header received
0810 18:47:16.687442 (+ 371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:16.687451 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:16.687545 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:16.687640 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:16.689941 (+ 2301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:16.691084 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:16.693002 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:16.693720 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:16.693823 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:16.697688 (+ 3865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:16.697699 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:16.697708 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:47:16.697763 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:16.703473 (+ 5710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:16.704395 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:16.704402 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:16.704406 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:16.704579 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:16.705067 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:16.705074 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:16.705078 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:16.705627 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:16.705932 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:16.706642 (+ 710us) server_negotiation.cc:299] Negotiation successful
0810 18:47:16.706886 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 18:47:16.728019 4465 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:47:16.729395 4465 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:47:16.739110 4465 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f8fd8f4a66f4005a91564f0b537a9b4: Generated new TSK 0
I20260810 18:47:16.739985 4465 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:47:16.747443 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 4330
I20260810 18:47:16.748863 4428 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:47:16.840010 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 4330
2026-08-10T18:47:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2389 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:47:16 dist-test-slave-7kzw krb5kdc[4489](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:47:16 dist-test-slave-7kzw krb5kdc[4489](info): set up 2 sockets
Aug 10 18:47:16 dist-test-slave-7kzw krb5kdc[4489](info): commencing operation
krb5kdc: starting...
W20260810 18:47:18.196745 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.308s user 0.001s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:47:18 dist-test-slave-7kzw krb5kdc[4489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:47:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:47:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:47:18.534531 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38107
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38107
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:47:18.907361 4506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:47:18.907697 4506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:47:18.907897 4506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:47:18.915462 4506 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:47:18.915591 4506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:47:18.915691 4506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:47:18.915791 4506 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:47:18.915882 4506 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:47:18.926155 4506 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38107
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:47:18.927706 4506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:47:18.929625 4506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:47:18.942389 4514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:18.942459 4516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:18.942368 4518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:47:18.943779 4506 server_base.cc:1034] running on GCE node
I20260810 18:47:18.944670 4506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:47:18.946236 4506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:47:18.947501 4506 hybrid_clock.cc:648] HybridClock initialized: now 1786387638947445 us; error 50 us; skew 500 ppm
I20260810 18:47:18.947883 4506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:47:18 dist-test-slave-7kzw krb5kdc[4489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387638, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:47:18.951519 4506 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:47:18.953234 4506 webserver.cc:460] Webserver started at http://127.0.0.1:36427/ using document root <none> and password file <none>
I20260810 18:47:18.954020 4506 fs_manager.cc:359] Metadata directory not provided
I20260810 18:47:18.954174 4506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:47:18.954581 4506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:47:18.957787 4506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "d8da338f0abc4720ae78c15580065ddc"
format_stamp: "Formatted at 2026-08-10 18:47:18 on dist-test-slave-7kzw"
server_key: "2b258eab31f858c7fb2cd10a82f36b95"
server_key_iv: "ee830ef707bf9d9151b65d5ba725fd1b"
server_key_version: "encryptionkey@0"
I20260810 18:47:18.958705 4506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "d8da338f0abc4720ae78c15580065ddc"
format_stamp: "Formatted at 2026-08-10 18:47:18 on dist-test-slave-7kzw"
server_key: "2b258eab31f858c7fb2cd10a82f36b95"
server_key_iv: "ee830ef707bf9d9151b65d5ba725fd1b"
server_key_version: "encryptionkey@0"
I20260810 18:47:18.964248 4506 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 18:47:18.967913 4527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:18.969329 4506 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:47:18.969538 4506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "d8da338f0abc4720ae78c15580065ddc"
format_stamp: "Formatted at 2026-08-10 18:47:18 on dist-test-slave-7kzw"
server_key: "2b258eab31f858c7fb2cd10a82f36b95"
server_key_iv: "ee830ef707bf9d9151b65d5ba725fd1b"
server_key_version: "encryptionkey@0"
I20260810 18:47:18.969749 4506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:47:19.000295 4506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:47:19.004957 4506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:47:19.005307 4506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:47:19.026134 4506 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38107
I20260810 18:47:19.026166 4601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38107 every 8 connection(s)
I20260810 18:47:19.027686 4506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:47:19.030251 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 4506
I20260810 18:47:19.030453 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:47:19.030946 5011 external_mini_cluster.cc:1428] Setting key 010fa4811bd272edd106fb20a8d941bf
I20260810 18:47:19.032400 4603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:47:19.045445 4607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:19.032999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32951 (local address 127.0.0.1:38107)
0810 18:47:19.033837 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:19.033849 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:19.033891 (+ 42us) server_negotiation.cc:407] Connection header received
0810 18:47:19.035521 (+ 1630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:19.035545 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:19.035951 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:19.036324 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:19.037010 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:19.038627 (+ 1617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:19.040668 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:19.041440 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:19.041605 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:19.042938 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:19.042949 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:19.042963 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:47:19.043557 (+ 594us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:19.043671 (+ 114us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:47:19.043702 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:19.043804 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:19.044102 (+ 298us) server_negotiation.cc:299] Negotiation successful
0810 18:47:19.044429 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":371,"threads_started":1}
I20260810 18:47:19.046293 4603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8da338f0abc4720ae78c15580065ddc: Bootstrap starting.
I20260810 18:47:19.049407 5011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:47:19.049996 4603 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8da338f0abc4720ae78c15580065ddc: Neither blocks nor log segments found. Creating new log.
I20260810 18:47:19.051120 4603 log.cc:824] T 00000000000000000000000000000000 P d8da338f0abc4720ae78c15580065ddc: Log is configured to *not* fsync() on all Append() calls
I20260810 18:47:19.054857 4603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8da338f0abc4720ae78c15580065ddc: No bootstrap required, opened a new log
I20260810 18:47:19.062006 4607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:19.054592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54913 (local address 127.0.0.1:38107)
0810 18:47:19.054782 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:19.054789 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:19.055267 (+ 478us) server_negotiation.cc:407] Connection header received
0810 18:47:19.055433 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:19.055439 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:19.055546 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:19.055681 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:19.056258 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:19.057445 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:19.059033 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:19.059813 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:19.059958 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:19.061098 (+ 1140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:19.061111 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:19.061118 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:47:19.061149 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:19.061202 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:47:19.061226 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:19.061337 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:19.061463 (+ 126us) server_negotiation.cc:299] Negotiation successful
0810 18:47:19.061590 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 18:47:19.065354 4603 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8da338f0abc4720ae78c15580065ddc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8da338f0abc4720ae78c15580065ddc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38107 } }
I20260810 18:47:19.065770 4603 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8da338f0abc4720ae78c15580065ddc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:47:19.065888 4603 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8da338f0abc4720ae78c15580065ddc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8da338f0abc4720ae78c15580065ddc, State: Initialized, Role: FOLLOWER
I20260810 18:47:19.066694 4603 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8da338f0abc4720ae78c15580065ddc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8da338f0abc4720ae78c15580065ddc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38107 } }
I20260810 18:47:19.066952 4603 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8da338f0abc4720ae78c15580065ddc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:47:19.067134 4603 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8da338f0abc4720ae78c15580065ddc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:47:19.067366 4603 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8da338f0abc4720ae78c15580065ddc [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:47:19.069775 4603 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8da338f0abc4720ae78c15580065ddc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8da338f0abc4720ae78c15580065ddc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38107 } }
I20260810 18:47:19.070386 4603 leader_election.cc:302] T 00000000000000000000000000000000 P d8da338f0abc4720ae78c15580065ddc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8da338f0abc4720ae78c15580065ddc; no voters:
I20260810 18:47:19.071116 4603 leader_election.cc:288] T 00000000000000000000000000000000 P d8da338f0abc4720ae78c15580065ddc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:47:19.071271 4619 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8da338f0abc4720ae78c15580065ddc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:47:19.072050 4619 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8da338f0abc4720ae78c15580065ddc [term 1 LEADER]: Becoming Leader. State: Replica: d8da338f0abc4720ae78c15580065ddc, State: Running, Role: LEADER
I20260810 18:47:19.072597 4619 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8da338f0abc4720ae78c15580065ddc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8da338f0abc4720ae78c15580065ddc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38107 } }
I20260810 18:47:19.073531 4603 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8da338f0abc4720ae78c15580065ddc [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:47:19.075896 4620 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8da338f0abc4720ae78c15580065ddc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8da338f0abc4720ae78c15580065ddc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8da338f0abc4720ae78c15580065ddc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38107 } } }
I20260810 18:47:19.076364 4620 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8da338f0abc4720ae78c15580065ddc [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:19.079082 4621 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8da338f0abc4720ae78c15580065ddc [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8da338f0abc4720ae78c15580065ddc. Latest consensus state: current_term: 1 leader_uuid: "d8da338f0abc4720ae78c15580065ddc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8da338f0abc4720ae78c15580065ddc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38107 } } }
I20260810 18:47:19.079504 4621 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8da338f0abc4720ae78c15580065ddc [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:19.080602 4629 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:47:19.085250 4629 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:47:19.096738 4629 catalog_manager.cc:1347] Generated new cluster ID: b2cc0dbfb69540ee89239b9459cdd256
I20260810 18:47:19.096868 4629 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:47:19.108343 4629 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:47:19.110261 4629 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:47:19.119828 4629 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8da338f0abc4720ae78c15580065ddc: Generated new TSK 0
I20260810 18:47:19.120889 4629 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:47:19.136089 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 4506
W20260810 18:47:19.137784 4597 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:32951
W20260810 18:47:19.138612 5011 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 4506: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:47:19.138768 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 4506
2026-08-10T18:47:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2296 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:47:19 dist-test-slave-7kzw krb5kdc[4650](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:47:19 dist-test-slave-7kzw krb5kdc[4650](info): set up 2 sockets
Aug 10 18:47:19 dist-test-slave-7kzw krb5kdc[4650](info): commencing operation
krb5kdc: starting...
W20260810 18:47:20.493458 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.308s user 0.001s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:47:20 dist-test-slave-7kzw krb5kdc[4650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:47:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:47:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:47:20.856559 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46357
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46357
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:47:21.233311 4674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:47:21.233654 4674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:47:21.233840 4674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:47:21.241608 4674 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:47:21.241760 4674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:47:21.241866 4674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:47:21.241968 4674 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:47:21.242066 4674 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:47:21.253037 4674 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46357
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:47:21.254598 4674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:47:21.256177 4674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:47:21.269308 4680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:21.269363 4685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:21.269389 4682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:47:21.270576 4674 server_base.cc:1034] running on GCE node
I20260810 18:47:21.271500 4674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:47:21.273278 4674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:47:21.274655 4674 hybrid_clock.cc:648] HybridClock initialized: now 1786387641274571 us; error 66 us; skew 500 ppm
I20260810 18:47:21.275113 4674 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:47:21 dist-test-slave-7kzw krb5kdc[4650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387641, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:47:21.279170 4674 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:47:21.281080 4674 webserver.cc:460] Webserver started at http://127.0.0.1:40181/ using document root <none> and password file <none>
I20260810 18:47:21.281968 4674 fs_manager.cc:359] Metadata directory not provided
I20260810 18:47:21.282153 4674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:47:21.282619 4674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:47:21.286932 4674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "dbf60a8912004ba9bfe2e171b8c46e14"
format_stamp: "Formatted at 2026-08-10 18:47:21 on dist-test-slave-7kzw"
server_key: "64bef666b8863799b71aea5fd0867dd1"
server_key_iv: "ddcbc318f78d630f98e1e08b5fd3bc17"
server_key_version: "encryptionkey@0"
I20260810 18:47:21.287987 4674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "dbf60a8912004ba9bfe2e171b8c46e14"
format_stamp: "Formatted at 2026-08-10 18:47:21 on dist-test-slave-7kzw"
server_key: "64bef666b8863799b71aea5fd0867dd1"
server_key_iv: "ddcbc318f78d630f98e1e08b5fd3bc17"
server_key_version: "encryptionkey@0"
I20260810 18:47:21.294545 4674 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260810 18:47:21.298966 4692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:21.300832 4674 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260810 18:47:21.301079 4674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "dbf60a8912004ba9bfe2e171b8c46e14"
format_stamp: "Formatted at 2026-08-10 18:47:21 on dist-test-slave-7kzw"
server_key: "64bef666b8863799b71aea5fd0867dd1"
server_key_iv: "ddcbc318f78d630f98e1e08b5fd3bc17"
server_key_version: "encryptionkey@0"
I20260810 18:47:21.301344 4674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:47:21.307926 4674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:47:21.312515 4674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:47:21.312839 4674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:47:21.333854 4674 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46357
I20260810 18:47:21.333938 4770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46357 every 8 connection(s)
I20260810 18:47:21.335512 4674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:47:21.340312 4771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:47:21.343849 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 4674
I20260810 18:47:21.344129 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:47:21.344642 5011 external_mini_cluster.cc:1428] Setting key 4e94dc4c92ac1db39d30c075faac57fb
I20260810 18:47:21.353783 4775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:21.346934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36787 (local address 127.0.0.1:46357)
0810 18:47:21.347921 (+ 987us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:21.347935 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:21.347980 (+ 45us) server_negotiation.cc:407] Connection header received
0810 18:47:21.349789 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:21.349815 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:21.350203 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:21.350728 (+ 525us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:21.351988 (+ 1260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:21.351995 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:21.352009 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:47:21.352025 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:21.352095 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:47:21.352128 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:21.352383 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:21.352532 (+ 149us) server_negotiation.cc:299] Negotiation successful
0810 18:47:21.352754 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":753,"thread_start_us":437,"threads_started":1}
I20260810 18:47:21.354596 4771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbf60a8912004ba9bfe2e171b8c46e14: Bootstrap starting.
I20260810 18:47:21.358166 5011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:47:21.358788 4771 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbf60a8912004ba9bfe2e171b8c46e14: Neither blocks nor log segments found. Creating new log.
I20260810 18:47:21.359947 4771 log.cc:824] T 00000000000000000000000000000000 P dbf60a8912004ba9bfe2e171b8c46e14: Log is configured to *not* fsync() on all Append() calls
I20260810 18:47:21.363875 4771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbf60a8912004ba9bfe2e171b8c46e14: No bootstrap required, opened a new log
I20260810 18:47:21.366161 4775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:21.363311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57917 (local address 127.0.0.1:46357)
0810 18:47:21.363515 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:21.363524 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:21.363928 (+ 404us) server_negotiation.cc:407] Connection header received
0810 18:47:21.364184 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:21.364188 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:21.364237 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:21.364358 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:21.365415 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:21.365420 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:21.365427 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:47:21.365450 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:21.365492 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:47:21.365509 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:21.365668 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:21.365764 (+ 96us) server_negotiation.cc:299] Negotiation successful
0810 18:47:21.365845 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 18:47:21.373212 4771 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbf60a8912004ba9bfe2e171b8c46e14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf60a8912004ba9bfe2e171b8c46e14" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46357 } }
I20260810 18:47:21.373709 4771 raft_consensus.cc:374] T 00000000000000000000000000000000 P dbf60a8912004ba9bfe2e171b8c46e14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:47:21.373834 4771 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbf60a8912004ba9bfe2e171b8c46e14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbf60a8912004ba9bfe2e171b8c46e14, State: Initialized, Role: FOLLOWER
I20260810 18:47:21.374585 4771 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbf60a8912004ba9bfe2e171b8c46e14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf60a8912004ba9bfe2e171b8c46e14" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46357 } }
I20260810 18:47:21.374879 4771 raft_consensus.cc:388] T 00000000000000000000000000000000 P dbf60a8912004ba9bfe2e171b8c46e14 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:47:21.375046 4771 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbf60a8912004ba9bfe2e171b8c46e14 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:47:21.375250 4771 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbf60a8912004ba9bfe2e171b8c46e14 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:47:21.378211 4771 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbf60a8912004ba9bfe2e171b8c46e14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf60a8912004ba9bfe2e171b8c46e14" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46357 } }
I20260810 18:47:21.378887 4771 leader_election.cc:302] T 00000000000000000000000000000000 P dbf60a8912004ba9bfe2e171b8c46e14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dbf60a8912004ba9bfe2e171b8c46e14; no voters:
I20260810 18:47:21.379572 4771 leader_election.cc:288] T 00000000000000000000000000000000 P dbf60a8912004ba9bfe2e171b8c46e14 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:47:21.379796 4785 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbf60a8912004ba9bfe2e171b8c46e14 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:47:21.380713 4785 raft_consensus.cc:686] T 00000000000000000000000000000000 P dbf60a8912004ba9bfe2e171b8c46e14 [term 1 LEADER]: Becoming Leader. State: Replica: dbf60a8912004ba9bfe2e171b8c46e14, State: Running, Role: LEADER
I20260810 18:47:21.381439 4785 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbf60a8912004ba9bfe2e171b8c46e14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf60a8912004ba9bfe2e171b8c46e14" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46357 } }
I20260810 18:47:21.381960 4771 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbf60a8912004ba9bfe2e171b8c46e14 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:47:21.384984 4787 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbf60a8912004ba9bfe2e171b8c46e14 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbf60a8912004ba9bfe2e171b8c46e14. Latest consensus state: current_term: 1 leader_uuid: "dbf60a8912004ba9bfe2e171b8c46e14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf60a8912004ba9bfe2e171b8c46e14" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46357 } } }
I20260810 18:47:21.385548 4787 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbf60a8912004ba9bfe2e171b8c46e14 [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:21.385002 4786 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbf60a8912004ba9bfe2e171b8c46e14 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dbf60a8912004ba9bfe2e171b8c46e14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbf60a8912004ba9bfe2e171b8c46e14" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46357 } } }
I20260810 18:47:21.386073 4786 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbf60a8912004ba9bfe2e171b8c46e14 [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:21.388799 4793 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:47:21.395646 4793 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:47:21.407588 4793 catalog_manager.cc:1347] Generated new cluster ID: 5fa73c5dfd0c48c682a2f41c5fb543cb
I20260810 18:47:21.407747 4793 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:47:21.423574 4793 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:47:21.424105 4793 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:47:21.440573 4793 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dbf60a8912004ba9bfe2e171b8c46e14: Generated new TSK 0
I20260810 18:47:21.441802 4793 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:47:21.450930 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 4674
W20260810 18:47:21.452196 4766 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:36787
W20260810 18:47:21.453078 5011 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 4674: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:47:21.453272 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 4674
2026-08-10T18:47:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2321 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:47:21 dist-test-slave-7kzw krb5kdc[4813](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:47:21 dist-test-slave-7kzw krb5kdc[4813](info): set up 2 sockets
Aug 10 18:47:21 dist-test-slave-7kzw krb5kdc[4813](info): commencing operation
krb5kdc: starting...
W20260810 18:47:22.838898 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.331s user 0.003s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:47:23 dist-test-slave-7kzw krb5kdc[4813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:47:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:47:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:47:23.176175 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39963
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39963
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:47:23.543840 4834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:47:23.544174 4834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:47:23.544281 4834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:47:23.551970 4834 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:47:23.552098 4834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:47:23.552150 4834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:47:23.552217 4834 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:47:23.552295 4834 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:47:23.562812 4834 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39963
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:47:23.564211 4834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:47:23.565623 4834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:47:23.578390 4842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:23.578441 4843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:23.578497 4845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:47:23.579658 4834 server_base.cc:1034] running on GCE node
I20260810 18:47:23.580479 4834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:47:23.582036 4834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:47:23.583271 4834 hybrid_clock.cc:648] HybridClock initialized: now 1786387643583226 us; error 63 us; skew 500 ppm
I20260810 18:47:23.583647 4834 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:47:23 dist-test-slave-7kzw krb5kdc[4813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387643, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:47:23.591306 4834 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:47:23.592846 4834 webserver.cc:460] Webserver started at http://127.0.0.1:45009/ using document root <none> and password file <none>
I20260810 18:47:23.593547 4834 fs_manager.cc:359] Metadata directory not provided
I20260810 18:47:23.593650 4834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:47:23.593921 4834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:47:23.597262 4834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "9123f8a78e5a43fabb7d06534eb5dfce"
format_stamp: "Formatted at 2026-08-10 18:47:23 on dist-test-slave-7kzw"
server_key: "a4441742556e6d67c6ae1f86a57711dd"
server_key_iv: "ffa6dc542161f6baab8abdcb1722350a"
server_key_version: "encryptionkey@0"
I20260810 18:47:23.598047 4834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "9123f8a78e5a43fabb7d06534eb5dfce"
format_stamp: "Formatted at 2026-08-10 18:47:23 on dist-test-slave-7kzw"
server_key: "a4441742556e6d67c6ae1f86a57711dd"
server_key_iv: "ffa6dc542161f6baab8abdcb1722350a"
server_key_version: "encryptionkey@0"
I20260810 18:47:23.603433 4834 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 18:47:23.607784 4858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:23.609633 4834 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260810 18:47:23.609905 4834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "9123f8a78e5a43fabb7d06534eb5dfce"
format_stamp: "Formatted at 2026-08-10 18:47:23 on dist-test-slave-7kzw"
server_key: "a4441742556e6d67c6ae1f86a57711dd"
server_key_iv: "ffa6dc542161f6baab8abdcb1722350a"
server_key_version: "encryptionkey@0"
I20260810 18:47:23.610170 4834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:47:23.634975 4834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:47:23.641587 4834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:47:23.641950 4834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:47:23.664455 4834 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39963
I20260810 18:47:23.664480 4946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39963 every 8 connection(s)
I20260810 18:47:23.666613 4834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:47:23.670113 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 4834
I20260810 18:47:23.670325 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:47:23.671046 5011 external_mini_cluster.cc:1428] Setting key 8e6e3d687f44474dec8435ac8f5d3bf7
I20260810 18:47:23.671898 4948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:47:23.687975 4948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9123f8a78e5a43fabb7d06534eb5dfce: Bootstrap starting.
Aug 10 18:47:23 dist-test-slave-7kzw krb5kdc[4813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 18:47:23.691370 4948 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9123f8a78e5a43fabb7d06534eb5dfce: Neither blocks nor log segments found. Creating new log.
I20260810 18:47:23.692469 4948 log.cc:824] T 00000000000000000000000000000000 P 9123f8a78e5a43fabb7d06534eb5dfce: Log is configured to *not* fsync() on all Append() calls
I20260810 18:47:23.695564 4948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9123f8a78e5a43fabb7d06534eb5dfce: No bootstrap required, opened a new log
I20260810 18:47:23.697630 4955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:23.673040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:34163 (local address 127.0.0.1:39963)
0810 18:47:23.673964 (+ 924us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:23.673983 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:23.674043 (+ 60us) server_negotiation.cc:407] Connection header received
0810 18:47:23.677758 (+ 3715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:23.677790 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:23.678396 (+ 606us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:23.678938 (+ 542us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:23.680814 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:23.682061 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:23.683960 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:23.684654 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:23.684804 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:23.689834 (+ 5030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:23.689842 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:23.689856 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:47:23.689903 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:23.693307 (+ 3404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:23.694157 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:23.694169 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:23.694177 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:23.694392 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:23.694889 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:23.694894 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:23.694898 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:23.695420 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:23.695813 (+ 393us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:23.696337 (+ 524us) server_negotiation.cc:299] Negotiation successful
0810 18:47:23.696648 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":362,"threads_started":1}
I20260810 18:47:23.701493 5011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:47:23.703972 4948 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9123f8a78e5a43fabb7d06534eb5dfce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9123f8a78e5a43fabb7d06534eb5dfce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39963 } }
I20260810 18:47:23.704444 4948 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9123f8a78e5a43fabb7d06534eb5dfce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:47:23.704574 4948 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9123f8a78e5a43fabb7d06534eb5dfce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9123f8a78e5a43fabb7d06534eb5dfce, State: Initialized, Role: FOLLOWER
I20260810 18:47:23.705225 4948 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9123f8a78e5a43fabb7d06534eb5dfce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9123f8a78e5a43fabb7d06534eb5dfce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39963 } }
I20260810 18:47:23.705479 4948 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9123f8a78e5a43fabb7d06534eb5dfce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:47:23.705634 4948 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9123f8a78e5a43fabb7d06534eb5dfce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:47:23.705863 4948 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9123f8a78e5a43fabb7d06534eb5dfce [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:47:23.708379 4948 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9123f8a78e5a43fabb7d06534eb5dfce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9123f8a78e5a43fabb7d06534eb5dfce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39963 } }
I20260810 18:47:23.708954 4948 leader_election.cc:302] T 00000000000000000000000000000000 P 9123f8a78e5a43fabb7d06534eb5dfce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9123f8a78e5a43fabb7d06534eb5dfce; no voters:
I20260810 18:47:23.709651 4948 leader_election.cc:288] T 00000000000000000000000000000000 P 9123f8a78e5a43fabb7d06534eb5dfce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:47:23.709764 4967 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9123f8a78e5a43fabb7d06534eb5dfce [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:47:23.711233 4967 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9123f8a78e5a43fabb7d06534eb5dfce [term 1 LEADER]: Becoming Leader. State: Replica: 9123f8a78e5a43fabb7d06534eb5dfce, State: Running, Role: LEADER
I20260810 18:47:23.711879 4967 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9123f8a78e5a43fabb7d06534eb5dfce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9123f8a78e5a43fabb7d06534eb5dfce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39963 } }
I20260810 18:47:23.712277 4948 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9123f8a78e5a43fabb7d06534eb5dfce [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:47:23.714969 4969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9123f8a78e5a43fabb7d06534eb5dfce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9123f8a78e5a43fabb7d06534eb5dfce. Latest consensus state: current_term: 1 leader_uuid: "9123f8a78e5a43fabb7d06534eb5dfce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9123f8a78e5a43fabb7d06534eb5dfce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39963 } } }
I20260810 18:47:23.715400 4969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9123f8a78e5a43fabb7d06534eb5dfce [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:23.715955 4968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9123f8a78e5a43fabb7d06534eb5dfce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9123f8a78e5a43fabb7d06534eb5dfce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9123f8a78e5a43fabb7d06534eb5dfce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39963 } } }
I20260810 18:47:23.716267 4968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9123f8a78e5a43fabb7d06534eb5dfce [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:23.726122 4955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:23.705885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:51327 (local address 127.0.0.1:39963)
0810 18:47:23.706091 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:23.706100 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:23.706712 (+ 612us) server_negotiation.cc:407] Connection header received
0810 18:47:23.706907 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:23.706912 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:23.706991 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:23.707099 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:23.708522 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:23.709346 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:23.711014 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:23.711756 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:23.711893 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:23.715122 (+ 3229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:23.715128 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:23.715133 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:47:23.715191 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:23.720372 (+ 5181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:23.721709 (+ 1337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:23.721717 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:23.721724 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:23.721914 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:23.722675 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:23.722683 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:23.722689 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:23.723264 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:23.723644 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:23.725265 (+ 1621us) server_negotiation.cc:299] Negotiation successful
0810 18:47:23.725526 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 18:47:23.726389 4974 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:47:23.732527 4974 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:47:23.742599 4974 catalog_manager.cc:1347] Generated new cluster ID: e4dfeef889844e4b87bbfb1479279129
I20260810 18:47:23.742744 4974 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:47:23.756292 4974 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:47:23.758193 4974 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:47:23.768015 4974 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9123f8a78e5a43fabb7d06534eb5dfce: Generated new TSK 0
I20260810 18:47:23.769069 4974 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:47:23.775195 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 4834
I20260810 18:47:23.776482 4938 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:47:23.868700 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 4834
W20260810 18:47:23.896413 4810 connection.cc:535] client connection to 127.0.0.1:39963 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39963): Connection reset by peer (error 104)
2026-08-10T18:47:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2409 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:47:23 dist-test-slave-7kzw krb5kdc[5002](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:47:23 dist-test-slave-7kzw krb5kdc[5002](info): set up 2 sockets
Aug 10 18:47:23 dist-test-slave-7kzw krb5kdc[5002](info): commencing operation
krb5kdc: starting...
W20260810 18:47:25.244081 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.327s user 0.003s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:47:25 dist-test-slave-7kzw krb5kdc[5002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:47:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:47:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:47:25.586036 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34373
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34373
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:47:25.959249 5032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:47:25.959560 5032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:47:25.959656 5032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:47:25.966959 5032 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:47:25.967082 5032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:47:25.967129 5032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:47:25.967200 5032 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:47:25.967247 5032 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:47:25.977723 5032 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34373
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:47:25.979156 5032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:47:25.980470 5032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:47:25.993084 5038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:25.993120 5037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:25.993620 5040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:47:25.994038 5032 server_base.cc:1034] running on GCE node
I20260810 18:47:25.994971 5032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:47:25.996543 5032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:47:25.997803 5032 hybrid_clock.cc:648] HybridClock initialized: now 1786387645997743 us; error 42 us; skew 500 ppm
I20260810 18:47:25.998198 5032 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:47:25 dist-test-slave-7kzw krb5kdc[5002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387645, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:47:26.001873 5032 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:47:26.003602 5032 webserver.cc:460] Webserver started at http://127.0.0.1:38427/ using document root <none> and password file <none>
I20260810 18:47:26.004383 5032 fs_manager.cc:359] Metadata directory not provided
I20260810 18:47:26.004536 5032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:47:26.004897 5032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:47:26.008262 5032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "40a3d58f87054d56a23ae70a5e879981"
format_stamp: "Formatted at 2026-08-10 18:47:25 on dist-test-slave-7kzw"
server_key: "243ccb4e82b1de90060a5cede88ebf12"
server_key_iv: "0234cf43c02cfff975c1bfbb45cb9663"
server_key_version: "encryptionkey@0"
I20260810 18:47:26.009078 5032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "40a3d58f87054d56a23ae70a5e879981"
format_stamp: "Formatted at 2026-08-10 18:47:25 on dist-test-slave-7kzw"
server_key: "243ccb4e82b1de90060a5cede88ebf12"
server_key_iv: "0234cf43c02cfff975c1bfbb45cb9663"
server_key_version: "encryptionkey@0"
I20260810 18:47:26.014722 5032 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 18:47:26.018422 5051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:26.020057 5032 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:47:26.020256 5032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "40a3d58f87054d56a23ae70a5e879981"
format_stamp: "Formatted at 2026-08-10 18:47:25 on dist-test-slave-7kzw"
server_key: "243ccb4e82b1de90060a5cede88ebf12"
server_key_iv: "0234cf43c02cfff975c1bfbb45cb9663"
server_key_version: "encryptionkey@0"
I20260810 18:47:26.020462 5032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:47:26.052703 5032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:47:26.057356 5032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:47:26.057674 5032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:47:26.077510 5032 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34373
I20260810 18:47:26.077553 5128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34373 every 8 connection(s)
I20260810 18:47:26.079005 5032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:47:26.083221 5129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:47:26.083308 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 5032
I20260810 18:47:26.083526 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:47:26.083987 5011 external_mini_cluster.cc:1428] Setting key 0e16e164a89bf4ba2c2076c7c2a49538
I20260810 18:47:26.100041 5129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40a3d58f87054d56a23ae70a5e879981: Bootstrap starting.
I20260810 18:47:26.100209 5135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:26.085985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:41699 (local address 127.0.0.1:34373)
0810 18:47:26.086869 (+ 884us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:26.086885 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:26.086942 (+ 57us) server_negotiation.cc:407] Connection header received
0810 18:47:26.088854 (+ 1912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:26.088892 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:26.089449 (+ 557us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:26.089934 (+ 485us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:26.090544 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:26.092624 (+ 2080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:26.094024 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:26.095151 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:26.095377 (+ 226us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:26.096838 (+ 1461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:26.096854 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:26.096881 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:47:26.097746 (+ 865us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:26.097856 (+ 110us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:47:26.097891 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:26.098066 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:26.098542 (+ 476us) server_negotiation.cc:299] Negotiation successful
0810 18:47:26.098751 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":383,"threads_started":1}
I20260810 18:47:26.103869 5129 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40a3d58f87054d56a23ae70a5e879981: Neither blocks nor log segments found. Creating new log.
I20260810 18:47:26.105067 5011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:47:26.105144 5129 log.cc:824] T 00000000000000000000000000000000 P 40a3d58f87054d56a23ae70a5e879981: Log is configured to *not* fsync() on all Append() calls
I20260810 18:47:26.108246 5129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40a3d58f87054d56a23ae70a5e879981: No bootstrap required, opened a new log
I20260810 18:47:26.117137 5135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:26.109920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:45251 (local address 127.0.0.1:34373)
0810 18:47:26.110120 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:26.110126 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:26.110511 (+ 385us) server_negotiation.cc:407] Connection header received
0810 18:47:26.110698 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:26.110704 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:26.110798 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:26.110917 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:26.111415 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:26.112620 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:26.114008 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:26.114773 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:26.114911 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:26.116164 (+ 1253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:26.116172 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:26.116178 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:47:26.116221 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:26.116288 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:47:26.116309 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:26.116456 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:26.116715 (+ 259us) server_negotiation.cc:299] Negotiation successful
0810 18:47:26.116780 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 18:47:26.118283 5129 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40a3d58f87054d56a23ae70a5e879981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40a3d58f87054d56a23ae70a5e879981" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34373 } }
I20260810 18:47:26.118880 5129 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40a3d58f87054d56a23ae70a5e879981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:47:26.119012 5129 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40a3d58f87054d56a23ae70a5e879981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40a3d58f87054d56a23ae70a5e879981, State: Initialized, Role: FOLLOWER
I20260810 18:47:26.119614 5129 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40a3d58f87054d56a23ae70a5e879981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40a3d58f87054d56a23ae70a5e879981" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34373 } }
I20260810 18:47:26.119850 5129 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40a3d58f87054d56a23ae70a5e879981 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:47:26.119948 5129 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40a3d58f87054d56a23ae70a5e879981 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:47:26.120085 5129 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40a3d58f87054d56a23ae70a5e879981 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:47:26.122977 5129 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40a3d58f87054d56a23ae70a5e879981 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40a3d58f87054d56a23ae70a5e879981" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34373 } }
I20260810 18:47:26.123525 5129 leader_election.cc:302] T 00000000000000000000000000000000 P 40a3d58f87054d56a23ae70a5e879981 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40a3d58f87054d56a23ae70a5e879981; no voters:
I20260810 18:47:26.124231 5129 leader_election.cc:288] T 00000000000000000000000000000000 P 40a3d58f87054d56a23ae70a5e879981 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:47:26.124392 5148 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40a3d58f87054d56a23ae70a5e879981 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:47:26.125216 5148 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40a3d58f87054d56a23ae70a5e879981 [term 1 LEADER]: Becoming Leader. State: Replica: 40a3d58f87054d56a23ae70a5e879981, State: Running, Role: LEADER
I20260810 18:47:26.125872 5148 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40a3d58f87054d56a23ae70a5e879981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40a3d58f87054d56a23ae70a5e879981" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34373 } }
I20260810 18:47:26.126827 5129 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40a3d58f87054d56a23ae70a5e879981 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:47:26.129649 5149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40a3d58f87054d56a23ae70a5e879981 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40a3d58f87054d56a23ae70a5e879981" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40a3d58f87054d56a23ae70a5e879981" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34373 } } }
I20260810 18:47:26.129741 5151 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40a3d58f87054d56a23ae70a5e879981 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40a3d58f87054d56a23ae70a5e879981. Latest consensus state: current_term: 1 leader_uuid: "40a3d58f87054d56a23ae70a5e879981" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40a3d58f87054d56a23ae70a5e879981" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34373 } } }
I20260810 18:47:26.130152 5149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40a3d58f87054d56a23ae70a5e879981 [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:26.130149 5151 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40a3d58f87054d56a23ae70a5e879981 [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:26.137126 5160 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:47:26.143409 5160 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:47:26.152863 5160 catalog_manager.cc:1347] Generated new cluster ID: cfd314b5169e4c84be930626360129a7
I20260810 18:47:26.153028 5160 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:47:26.171423 5160 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:47:26.173338 5160 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:47:26.183495 5160 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40a3d58f87054d56a23ae70a5e879981: Generated new TSK 0
I20260810 18:47:26.184438 5160 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:47:26.200062 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 5032
W20260810 18:47:26.201328 5120 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.136:41699
W20260810 18:47:26.202311 5011 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 5032: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:47:26.202502 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 5032
W20260810 18:47:26.236694 4998 connection.cc:535] client connection to 127.0.0.1:34373 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34373): Connection reset by peer (error 104)
2026-08-10T18:47:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2339 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:47:26 dist-test-slave-7kzw krb5kdc[5184](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:47:26 dist-test-slave-7kzw krb5kdc[5184](info): set up 2 sockets
Aug 10 18:47:26 dist-test-slave-7kzw krb5kdc[5184](info): commencing operation
krb5kdc: starting...
W20260810 18:47:27.616072 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.360s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:47:27 dist-test-slave-7kzw krb5kdc[5184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:47:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:47:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:47:27.988086 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35753
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:47:28.369372 5202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:47:28.369751 5202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:47:28.369851 5202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:47:28.377568 5202 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:47:28.377720 5202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:47:28.377771 5202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:47:28.377828 5202 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:47:28.377866 5202 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:47:28.388455 5202 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:47:28.389953 5202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:47:28.391466 5202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:47:28.404789 5209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:28.404816 5212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:28.404870 5210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:47:28.405575 5202 server_base.cc:1034] running on GCE node
I20260810 18:47:28.406531 5202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:47:28.408224 5202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:47:28.409480 5202 hybrid_clock.cc:648] HybridClock initialized: now 1786387648409429 us; error 30 us; skew 500 ppm
I20260810 18:47:28.409886 5202 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:47:28 dist-test-slave-7kzw krb5kdc[5184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387648, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:47:28.416894 5202 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:47:28.418701 5202 webserver.cc:460] Webserver started at http://127.0.0.1:45785/ using document root <none> and password file <none>
I20260810 18:47:28.419459 5202 fs_manager.cc:359] Metadata directory not provided
I20260810 18:47:28.419574 5202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:47:28.420172 5202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:47:28.423549 5202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "59a7550b36834a4ba169c2d71e13b758"
format_stamp: "Formatted at 2026-08-10 18:47:28 on dist-test-slave-7kzw"
server_key: "865bf1c30d449b7ac96de38aaf4dfd60"
server_key_iv: "1f85b81992755c29ad9ccfcfe0116aa1"
server_key_version: "encryptionkey@0"
I20260810 18:47:28.424300 5202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "59a7550b36834a4ba169c2d71e13b758"
format_stamp: "Formatted at 2026-08-10 18:47:28 on dist-test-slave-7kzw"
server_key: "865bf1c30d449b7ac96de38aaf4dfd60"
server_key_iv: "1f85b81992755c29ad9ccfcfe0116aa1"
server_key_version: "encryptionkey@0"
I20260810 18:47:28.430135 5202 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 18:47:28.434152 5226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:28.435896 5202 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:47:28.436136 5202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "59a7550b36834a4ba169c2d71e13b758"
format_stamp: "Formatted at 2026-08-10 18:47:28 on dist-test-slave-7kzw"
server_key: "865bf1c30d449b7ac96de38aaf4dfd60"
server_key_iv: "1f85b81992755c29ad9ccfcfe0116aa1"
server_key_version: "encryptionkey@0"
I20260810 18:47:28.436370 5202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:47:28.442919 5202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:47:28.447739 5202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:47:28.448086 5202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:47:28.469511 5202 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35753
I20260810 18:47:28.469616 5295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35753 every 8 connection(s)
I20260810 18:47:28.471460 5202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:47:28.475963 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 5202
I20260810 18:47:28.476190 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:47:28.476824 5297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:47:28.476859 5011 external_mini_cluster.cc:1428] Setting key ac71dbe9276eb150e347c9a08567d74a
I20260810 18:47:28.485841 5304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:28.479056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:47143 (local address 127.0.0.1:35753)
0810 18:47:28.480019 (+ 963us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:28.480035 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:28.480084 (+ 49us) server_negotiation.cc:407] Connection header received
0810 18:47:28.482042 (+ 1958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:28.482068 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:28.482444 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:28.482918 (+ 474us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:28.484127 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:28.484137 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:28.484156 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:47:28.484171 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:28.484236 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:47:28.484262 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:28.484455 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:28.484559 (+ 104us) server_negotiation.cc:299] Negotiation successful
0810 18:47:28.484838 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":726,"thread_start_us":407,"threads_started":1}
I20260810 18:47:28.489946 5011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:47:28.491884 5297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59a7550b36834a4ba169c2d71e13b758: Bootstrap starting.
I20260810 18:47:28.496469 5297 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59a7550b36834a4ba169c2d71e13b758: Neither blocks nor log segments found. Creating new log.
I20260810 18:47:28.498003 5297 log.cc:824] T 00000000000000000000000000000000 P 59a7550b36834a4ba169c2d71e13b758: Log is configured to *not* fsync() on all Append() calls
I20260810 18:47:28.498428 5304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:28.495606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:39715 (local address 127.0.0.1:35753)
0810 18:47:28.495832 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:28.495838 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:28.496347 (+ 509us) server_negotiation.cc:407] Connection header received
0810 18:47:28.496512 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:28.496518 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:28.496593 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:28.496726 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:28.497702 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:28.497712 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:28.497720 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:47:28.497745 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:28.497808 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:47:28.497829 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:28.497948 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:28.498068 (+ 120us) server_negotiation.cc:299] Negotiation successful
0810 18:47:28.498137 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260810 18:47:28.505330 5297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59a7550b36834a4ba169c2d71e13b758: No bootstrap required, opened a new log
I20260810 18:47:28.513346 5297 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59a7550b36834a4ba169c2d71e13b758 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59a7550b36834a4ba169c2d71e13b758" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35753 } }
I20260810 18:47:28.513842 5297 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59a7550b36834a4ba169c2d71e13b758 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:47:28.513983 5297 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59a7550b36834a4ba169c2d71e13b758 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59a7550b36834a4ba169c2d71e13b758, State: Initialized, Role: FOLLOWER
I20260810 18:47:28.514736 5297 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59a7550b36834a4ba169c2d71e13b758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59a7550b36834a4ba169c2d71e13b758" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35753 } }
I20260810 18:47:28.515019 5297 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59a7550b36834a4ba169c2d71e13b758 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:47:28.515195 5297 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59a7550b36834a4ba169c2d71e13b758 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:47:28.515410 5297 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59a7550b36834a4ba169c2d71e13b758 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:47:28.518189 5297 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59a7550b36834a4ba169c2d71e13b758 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59a7550b36834a4ba169c2d71e13b758" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35753 } }
I20260810 18:47:28.518828 5297 leader_election.cc:302] T 00000000000000000000000000000000 P 59a7550b36834a4ba169c2d71e13b758 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59a7550b36834a4ba169c2d71e13b758; no voters:
I20260810 18:47:28.519520 5297 leader_election.cc:288] T 00000000000000000000000000000000 P 59a7550b36834a4ba169c2d71e13b758 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:47:28.519714 5313 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59a7550b36834a4ba169c2d71e13b758 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:47:28.520438 5313 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59a7550b36834a4ba169c2d71e13b758 [term 1 LEADER]: Becoming Leader. State: Replica: 59a7550b36834a4ba169c2d71e13b758, State: Running, Role: LEADER
I20260810 18:47:28.521070 5313 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59a7550b36834a4ba169c2d71e13b758 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59a7550b36834a4ba169c2d71e13b758" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35753 } }
I20260810 18:47:28.522198 5297 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59a7550b36834a4ba169c2d71e13b758 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:47:28.524925 5315 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59a7550b36834a4ba169c2d71e13b758 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59a7550b36834a4ba169c2d71e13b758. Latest consensus state: current_term: 1 leader_uuid: "59a7550b36834a4ba169c2d71e13b758" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59a7550b36834a4ba169c2d71e13b758" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35753 } } }
I20260810 18:47:28.525341 5315 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59a7550b36834a4ba169c2d71e13b758 [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:28.526290 5332 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:47:28.524839 5314 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59a7550b36834a4ba169c2d71e13b758 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59a7550b36834a4ba169c2d71e13b758" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59a7550b36834a4ba169c2d71e13b758" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35753 } } }
I20260810 18:47:28.526818 5314 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59a7550b36834a4ba169c2d71e13b758 [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:28.534143 5332 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:47:28.545056 5332 catalog_manager.cc:1347] Generated new cluster ID: d6c4d8450832437fa623bcc75e6e35b5
I20260810 18:47:28.545248 5332 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:47:28.564903 5332 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:47:28.565501 5332 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:47:28.580694 5332 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59a7550b36834a4ba169c2d71e13b758: Generated new TSK 0
I20260810 18:47:28.581835 5332 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:47:28.592142 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 5202
W20260810 18:47:28.593557 5290 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.136:47143
W20260810 18:47:28.594410 5011 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 5202: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:47:28.594610 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 5202
2026-08-10T18:47:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2395 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:47:28 dist-test-slave-7kzw krb5kdc[5352](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:47:28 dist-test-slave-7kzw krb5kdc[5352](info): set up 2 sockets
Aug 10 18:47:28 dist-test-slave-7kzw krb5kdc[5352](info): commencing operation
krb5kdc: starting...
W20260810 18:47:29.986042 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.334s user 0.006s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:47:30 dist-test-slave-7kzw krb5kdc[5352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:47:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:47:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:47:30.361937 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42533
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42533
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:47:30.878674 5368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:47:30.879040 5368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:47:30.879206 5368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:47:30.886562 5368 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:47:30.886673 5368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:47:30.886756 5368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:47:30.886840 5368 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:47:30.886904 5368 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:47:30.896942 5368 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42533
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:47:30.898926 5368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:47:30.900496 5368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:47:30.913527 5380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:30.913609 5377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:30.913582 5383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:47:30.914600 5368 server_base.cc:1034] running on GCE node
I20260810 18:47:30.915441 5368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:47:30.916955 5368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:47:30.918169 5368 hybrid_clock.cc:648] HybridClock initialized: now 1786387650918124 us; error 33 us; skew 500 ppm
I20260810 18:47:30.918624 5368 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:47:30 dist-test-slave-7kzw krb5kdc[5352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387650, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:47:30.921984 5368 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:47:30.923671 5368 webserver.cc:460] Webserver started at http://127.0.0.1:42637/ using document root <none> and password file <none>
I20260810 18:47:30.924412 5368 fs_manager.cc:359] Metadata directory not provided
I20260810 18:47:30.924564 5368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:47:30.925061 5368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:47:30.928789 5368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "4d2552e68db74db8bd263c13a82f900d"
format_stamp: "Formatted at 2026-08-10 18:47:30 on dist-test-slave-7kzw"
server_key: "56280f2b7cc503dc08896b181c6b34a0"
server_key_iv: "114bdb79c084ae2b2eb75076bd91cf7f"
server_key_version: "encryptionkey@0"
I20260810 18:47:30.929697 5368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "4d2552e68db74db8bd263c13a82f900d"
format_stamp: "Formatted at 2026-08-10 18:47:30 on dist-test-slave-7kzw"
server_key: "56280f2b7cc503dc08896b181c6b34a0"
server_key_iv: "114bdb79c084ae2b2eb75076bd91cf7f"
server_key_version: "encryptionkey@0"
I20260810 18:47:30.935956 5368 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260810 18:47:30.939845 5399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:30.941248 5368 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260810 18:47:30.941448 5368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "4d2552e68db74db8bd263c13a82f900d"
format_stamp: "Formatted at 2026-08-10 18:47:30 on dist-test-slave-7kzw"
server_key: "56280f2b7cc503dc08896b181c6b34a0"
server_key_iv: "114bdb79c084ae2b2eb75076bd91cf7f"
server_key_version: "encryptionkey@0"
I20260810 18:47:30.941701 5368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:47:30.966141 5368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:47:30.971312 5368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:47:30.971580 5368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:47:30.991698 5368 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42533
I20260810 18:47:30.991741 5469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42533 every 8 connection(s)
I20260810 18:47:30.993172 5368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:47:30.997354 5470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:47:30.998729 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 5368
I20260810 18:47:30.998929 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:47:30.999397 5011 external_mini_cluster.cc:1428] Setting key 7c02250156ef29f622a3413236411e8a
I20260810 18:47:31.012850 5470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d2552e68db74db8bd263c13a82f900d: Bootstrap starting.
Aug 10 18:47:31 dist-test-slave-7kzw krb5kdc[5352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 18:47:31.016546 5470 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d2552e68db74db8bd263c13a82f900d: Neither blocks nor log segments found. Creating new log.
I20260810 18:47:31.018026 5470 log.cc:824] T 00000000000000000000000000000000 P 4d2552e68db74db8bd263c13a82f900d: Log is configured to *not* fsync() on all Append() calls
I20260810 18:47:31.021728 5470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d2552e68db74db8bd263c13a82f900d: No bootstrap required, opened a new log
I20260810 18:47:31.025904 5473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:31.001515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:40819 (local address 127.0.0.1:42533)
0810 18:47:31.002371 (+ 856us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:31.002388 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:31.002444 (+ 56us) server_negotiation.cc:407] Connection header received
0810 18:47:31.004417 (+ 1973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:31.004451 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:31.005038 (+ 587us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:31.005545 (+ 507us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:31.007042 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:31.009148 (+ 2106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:31.010623 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:31.011724 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:31.011934 (+ 210us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:31.016627 (+ 4693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:31.016641 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:31.016661 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:47:31.016717 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:31.020833 (+ 4116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:31.021655 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:31.021663 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:31.021673 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:31.021931 (+ 258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:31.022534 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:31.022543 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:31.022548 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:31.023293 (+ 745us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:31.023665 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:31.024026 (+ 361us) server_negotiation.cc:299] Negotiation successful
0810 18:47:31.024466 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":343,"threads_started":1}
I20260810 18:47:31.030421 5011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:47:31.031796 5470 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d2552e68db74db8bd263c13a82f900d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d2552e68db74db8bd263c13a82f900d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42533 } }
I20260810 18:47:31.032207 5470 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d2552e68db74db8bd263c13a82f900d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:47:31.032321 5470 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d2552e68db74db8bd263c13a82f900d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d2552e68db74db8bd263c13a82f900d, State: Initialized, Role: FOLLOWER
I20260810 18:47:31.032968 5470 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d2552e68db74db8bd263c13a82f900d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d2552e68db74db8bd263c13a82f900d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42533 } }
I20260810 18:47:31.033238 5470 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d2552e68db74db8bd263c13a82f900d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:47:31.033401 5470 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d2552e68db74db8bd263c13a82f900d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:47:31.033607 5470 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d2552e68db74db8bd263c13a82f900d [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:47:31.036616 5470 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d2552e68db74db8bd263c13a82f900d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d2552e68db74db8bd263c13a82f900d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42533 } }
I20260810 18:47:31.037286 5470 leader_election.cc:302] T 00000000000000000000000000000000 P 4d2552e68db74db8bd263c13a82f900d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d2552e68db74db8bd263c13a82f900d; no voters:
I20260810 18:47:31.038028 5470 leader_election.cc:288] T 00000000000000000000000000000000 P 4d2552e68db74db8bd263c13a82f900d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:47:31.038106 5486 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d2552e68db74db8bd263c13a82f900d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:47:31.039742 5486 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d2552e68db74db8bd263c13a82f900d [term 1 LEADER]: Becoming Leader. State: Replica: 4d2552e68db74db8bd263c13a82f900d, State: Running, Role: LEADER
I20260810 18:47:31.040382 5486 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d2552e68db74db8bd263c13a82f900d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d2552e68db74db8bd263c13a82f900d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42533 } }
I20260810 18:47:31.040572 5470 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d2552e68db74db8bd263c13a82f900d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:47:31.043275 5487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d2552e68db74db8bd263c13a82f900d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d2552e68db74db8bd263c13a82f900d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d2552e68db74db8bd263c13a82f900d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42533 } } }
I20260810 18:47:31.043713 5487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d2552e68db74db8bd263c13a82f900d [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:31.046025 5488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d2552e68db74db8bd263c13a82f900d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d2552e68db74db8bd263c13a82f900d. Latest consensus state: current_term: 1 leader_uuid: "4d2552e68db74db8bd263c13a82f900d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d2552e68db74db8bd263c13a82f900d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42533 } } }
I20260810 18:47:31.046504 5488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d2552e68db74db8bd263c13a82f900d [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:31.048314 5495 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:47:31.054888 5495 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:47:31.056116 5473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:31.035181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:49639 (local address 127.0.0.1:42533)
0810 18:47:31.035410 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:31.035421 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:31.035899 (+ 478us) server_negotiation.cc:407] Connection header received
0810 18:47:31.036133 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:31.036138 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:31.036209 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:31.036299 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:31.038592 (+ 2293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:31.039797 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:31.041679 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:31.042526 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:31.046631 (+ 4105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:31.046968 (+ 337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:31.046978 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:31.046986 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:47:31.047034 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:31.051535 (+ 4501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:31.052635 (+ 1100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:31.052640 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:31.052643 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:31.052829 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:31.053540 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:31.053545 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:31.053550 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:31.053946 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:31.055306 (+ 1360us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:31.055491 (+ 185us) server_negotiation.cc:299] Negotiation successful
0810 18:47:31.055680 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 18:47:31.065109 5495 catalog_manager.cc:1347] Generated new cluster ID: 82b8028a2100457197b69c495fe7ee15
I20260810 18:47:31.065239 5495 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:47:31.084836 5495 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:47:31.086308 5495 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:47:31.107296 5495 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d2552e68db74db8bd263c13a82f900d: Generated new TSK 0
I20260810 18:47:31.108258 5495 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:47:31.117564 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 5368
I20260810 18:47:31.119343 5465 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:47:31.213107 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 5368
W20260810 18:47:31.241636 5349 connection.cc:535] client connection to 127.0.0.1:42533 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42533): Connection reset by peer (error 104)
2026-08-10T18:47:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2610 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:47:31 dist-test-slave-7kzw krb5kdc[5520](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:47:31 dist-test-slave-7kzw krb5kdc[5520](info): set up 2 sockets
Aug 10 18:47:31 dist-test-slave-7kzw krb5kdc[5520](info): commencing operation
krb5kdc: starting...
W20260810 18:47:32.578835 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.317s user 0.003s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:47:32 dist-test-slave-7kzw krb5kdc[5520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:47:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:47:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:47:32.946357 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37249
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37249
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:47:33.313354 5541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:47:33.313738 5541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:47:33.313933 5541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:47:33.321482 5541 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:47:33.321647 5541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:47:33.321761 5541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:47:33.321877 5541 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:47:33.321960 5541 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:47:33.332096 5541 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37249
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:47:33.333551 5541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:47:33.335004 5541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:47:33.347666 5551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:33.347625 5555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:33.347702 5553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:47:33.349367 5541 server_base.cc:1034] running on GCE node
I20260810 18:47:33.350340 5541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:47:33.352142 5541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:47:33.353428 5541 hybrid_clock.cc:648] HybridClock initialized: now 1786387653353372 us; error 59 us; skew 500 ppm
I20260810 18:47:33.353847 5541 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:47:33 dist-test-slave-7kzw krb5kdc[5520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387653, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:47:33.357621 5541 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:47:33.359380 5541 webserver.cc:460] Webserver started at http://127.0.0.1:37675/ using document root <none> and password file <none>
I20260810 18:47:33.360189 5541 fs_manager.cc:359] Metadata directory not provided
I20260810 18:47:33.360350 5541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:47:33.360702 5541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:47:33.364051 5541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "d00d194e82db47bb96782a222636cdf9"
format_stamp: "Formatted at 2026-08-10 18:47:33 on dist-test-slave-7kzw"
server_key: "4af3ed3f1605ee470b7b3c3eb8c99b1b"
server_key_iv: "a87a7ad57428b1055cc363c52a861aa6"
server_key_version: "encryptionkey@0"
I20260810 18:47:33.364917 5541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "d00d194e82db47bb96782a222636cdf9"
format_stamp: "Formatted at 2026-08-10 18:47:33 on dist-test-slave-7kzw"
server_key: "4af3ed3f1605ee470b7b3c3eb8c99b1b"
server_key_iv: "a87a7ad57428b1055cc363c52a861aa6"
server_key_version: "encryptionkey@0"
I20260810 18:47:33.370781 5541 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 18:47:33.374496 5570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:33.375968 5541 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:47:33.376178 5541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "d00d194e82db47bb96782a222636cdf9"
format_stamp: "Formatted at 2026-08-10 18:47:33 on dist-test-slave-7kzw"
server_key: "4af3ed3f1605ee470b7b3c3eb8c99b1b"
server_key_iv: "a87a7ad57428b1055cc363c52a861aa6"
server_key_version: "encryptionkey@0"
I20260810 18:47:33.376399 5541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:47:33.423992 5541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:47:33.428839 5541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:47:33.429165 5541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:47:33.449121 5541 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37249
I20260810 18:47:33.449152 5648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37249 every 8 connection(s)
I20260810 18:47:33.450731 5541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:47:33.451656 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 5541
I20260810 18:47:33.451870 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:47:33.452432 5011 external_mini_cluster.cc:1428] Setting key 60d9c7153c2fc46d2151161492e3b131
I20260810 18:47:33.455657 5649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:47:33.467648 5653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:33.454793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:53381 (local address 127.0.0.1:37249)
0810 18:47:33.455654 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:33.455666 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:33.455713 (+ 47us) server_negotiation.cc:407] Connection header received
0810 18:47:33.457439 (+ 1726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:33.457461 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:33.457854 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:33.458203 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:33.458897 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:33.460261 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:33.462201 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:33.463024 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:33.463182 (+ 158us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:33.464706 (+ 1524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:33.464715 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:33.464730 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:47:33.465351 (+ 621us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:33.465434 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:47:33.465461 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:33.465605 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:33.466172 (+ 567us) server_negotiation.cc:299] Negotiation successful
0810 18:47:33.466415 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":376,"threads_started":1}
I20260810 18:47:33.469080 5649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d00d194e82db47bb96782a222636cdf9: Bootstrap starting.
I20260810 18:47:33.472003 5011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:47:33.472731 5649 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d00d194e82db47bb96782a222636cdf9: Neither blocks nor log segments found. Creating new log.
I20260810 18:47:33.473815 5649 log.cc:824] T 00000000000000000000000000000000 P d00d194e82db47bb96782a222636cdf9: Log is configured to *not* fsync() on all Append() calls
I20260810 18:47:33.476887 5649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d00d194e82db47bb96782a222636cdf9: No bootstrap required, opened a new log
I20260810 18:47:33.484287 5653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:33.476717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:57519 (local address 127.0.0.1:37249)
0810 18:47:33.476926 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:33.476934 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:33.477372 (+ 438us) server_negotiation.cc:407] Connection header received
0810 18:47:33.477538 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:33.477543 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:33.477642 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:33.477755 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:33.478270 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:33.479531 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:33.481064 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:33.481928 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:33.482066 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:33.483406 (+ 1340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:33.483413 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:33.483419 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:47:33.483446 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:33.483510 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:47:33.483549 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:33.483683 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:33.483858 (+ 175us) server_negotiation.cc:299] Negotiation successful
0810 18:47:33.483921 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 18:47:33.487350 5649 raft_consensus.cc:348] T 00000000000000000000000000000000 P d00d194e82db47bb96782a222636cdf9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d00d194e82db47bb96782a222636cdf9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37249 } }
I20260810 18:47:33.487757 5649 raft_consensus.cc:374] T 00000000000000000000000000000000 P d00d194e82db47bb96782a222636cdf9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:47:33.487838 5649 raft_consensus.cc:729] T 00000000000000000000000000000000 P d00d194e82db47bb96782a222636cdf9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d00d194e82db47bb96782a222636cdf9, State: Initialized, Role: FOLLOWER
I20260810 18:47:33.488477 5649 consensus_queue.cc:260] T 00000000000000000000000000000000 P d00d194e82db47bb96782a222636cdf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d00d194e82db47bb96782a222636cdf9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37249 } }
I20260810 18:47:33.488708 5649 raft_consensus.cc:388] T 00000000000000000000000000000000 P d00d194e82db47bb96782a222636cdf9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:47:33.488813 5649 raft_consensus.cc:482] T 00000000000000000000000000000000 P d00d194e82db47bb96782a222636cdf9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:47:33.488960 5649 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d00d194e82db47bb96782a222636cdf9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:47:33.491423 5649 raft_consensus.cc:504] T 00000000000000000000000000000000 P d00d194e82db47bb96782a222636cdf9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d00d194e82db47bb96782a222636cdf9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37249 } }
I20260810 18:47:33.491940 5649 leader_election.cc:302] T 00000000000000000000000000000000 P d00d194e82db47bb96782a222636cdf9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d00d194e82db47bb96782a222636cdf9; no voters:
I20260810 18:47:33.492561 5649 leader_election.cc:288] T 00000000000000000000000000000000 P d00d194e82db47bb96782a222636cdf9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:47:33.492749 5661 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d00d194e82db47bb96782a222636cdf9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:47:33.493444 5661 raft_consensus.cc:686] T 00000000000000000000000000000000 P d00d194e82db47bb96782a222636cdf9 [term 1 LEADER]: Becoming Leader. State: Replica: d00d194e82db47bb96782a222636cdf9, State: Running, Role: LEADER
I20260810 18:47:33.494093 5661 consensus_queue.cc:237] T 00000000000000000000000000000000 P d00d194e82db47bb96782a222636cdf9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d00d194e82db47bb96782a222636cdf9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37249 } }
I20260810 18:47:33.495025 5649 sys_catalog.cc:564] T 00000000000000000000000000000000 P d00d194e82db47bb96782a222636cdf9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:47:33.497658 5662 sys_catalog.cc:455] T 00000000000000000000000000000000 P d00d194e82db47bb96782a222636cdf9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d00d194e82db47bb96782a222636cdf9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d00d194e82db47bb96782a222636cdf9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37249 } } }
I20260810 18:47:33.498168 5662 sys_catalog.cc:458] T 00000000000000000000000000000000 P d00d194e82db47bb96782a222636cdf9 [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:33.497990 5664 sys_catalog.cc:455] T 00000000000000000000000000000000 P d00d194e82db47bb96782a222636cdf9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d00d194e82db47bb96782a222636cdf9. Latest consensus state: current_term: 1 leader_uuid: "d00d194e82db47bb96782a222636cdf9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d00d194e82db47bb96782a222636cdf9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37249 } } }
I20260810 18:47:33.498320 5664 sys_catalog.cc:458] T 00000000000000000000000000000000 P d00d194e82db47bb96782a222636cdf9 [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:33.501900 5672 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:47:33.507375 5672 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:47:33.516995 5672 catalog_manager.cc:1347] Generated new cluster ID: d610aea162624e0e9bd3d1967f3e4038
I20260810 18:47:33.517139 5672 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:47:33.536512 5672 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:47:33.538450 5672 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:47:33.553004 5672 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d00d194e82db47bb96782a222636cdf9: Generated new TSK 0
I20260810 18:47:33.554206 5672 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:47:33.566035 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 5541
W20260810 18:47:33.567169 5641 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.136:53381
W20260810 18:47:33.567996 5011 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 5541: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:47:33.568135 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 5541
W20260810 18:47:33.601893 5518 connection.cc:535] client connection to 127.0.0.1:37249 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37249): Connection reset by peer (error 104)
2026-08-10T18:47:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2359 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:47:33 dist-test-slave-7kzw krb5kdc[5686](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:47:33 dist-test-slave-7kzw krb5kdc[5686](info): set up 2 sockets
Aug 10 18:47:33 dist-test-slave-7kzw krb5kdc[5686](info): commencing operation
krb5kdc: starting...
W20260810 18:47:34.983680 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.362s user 0.000s sys 0.055s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:47:35 dist-test-slave-7kzw krb5kdc[5686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:47:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:47:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:47:35.339831 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33193
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33193
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:47:35.715480 5710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:47:35.715809 5710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:47:35.716029 5710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:47:35.724808 5710 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:47:35.724943 5710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:47:35.725055 5710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:47:35.725165 5710 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:47:35.725252 5710 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:47:35.736084 5710 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33193
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387267874959-5011-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:47:35.737545 5710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:47:35.739126 5710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:47:35.751895 5722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:35.752081 5715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:35.751920 5717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:47:35.753371 5710 server_base.cc:1034] running on GCE node
I20260810 18:47:35.754393 5710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:47:35.756139 5710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:47:35.757429 5710 hybrid_clock.cc:648] HybridClock initialized: now 1786387655757359 us; error 64 us; skew 500 ppm
I20260810 18:47:35.757858 5710 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:47:35 dist-test-slave-7kzw krb5kdc[5686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387655, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:47:35.761487 5710 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:47:35.763307 5710 webserver.cc:460] Webserver started at http://127.0.0.1:38477/ using document root <none> and password file <none>
I20260810 18:47:35.764072 5710 fs_manager.cc:359] Metadata directory not provided
I20260810 18:47:35.764237 5710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:47:35.764586 5710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:47:35.767925 5710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "3e3f5931bfd84108b43d542ace1cac75"
format_stamp: "Formatted at 2026-08-10 18:47:35 on dist-test-slave-7kzw"
server_key: "6d07520137bd84603bd2033e4dba99e0"
server_key_iv: "befa3b703f32f16262ba36dd2e847e12"
server_key_version: "encryptionkey@0"
I20260810 18:47:35.768756 5710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "3e3f5931bfd84108b43d542ace1cac75"
format_stamp: "Formatted at 2026-08-10 18:47:35 on dist-test-slave-7kzw"
server_key: "6d07520137bd84603bd2033e4dba99e0"
server_key_iv: "befa3b703f32f16262ba36dd2e847e12"
server_key_version: "encryptionkey@0"
I20260810 18:47:35.774688 5710 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 18:47:35.778891 5740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:35.780475 5710 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 18:47:35.780763 5710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "3e3f5931bfd84108b43d542ace1cac75"
format_stamp: "Formatted at 2026-08-10 18:47:35 on dist-test-slave-7kzw"
server_key: "6d07520137bd84603bd2033e4dba99e0"
server_key_iv: "befa3b703f32f16262ba36dd2e847e12"
server_key_version: "encryptionkey@0"
I20260810 18:47:35.781031 5710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:47:35.788239 5710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:47:35.793136 5710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:47:35.793515 5710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:47:35.814872 5710 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33193
I20260810 18:47:35.814917 5817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33193 every 8 connection(s)
I20260810 18:47:35.816632 5710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:47:35.817682 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 5710
I20260810 18:47:35.817888 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:47:35.818338 5011 external_mini_cluster.cc:1428] Setting key 472d782b1d97ae4a11f829146790b3ca
I20260810 18:47:35.821699 5819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:47:35.826915 5823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:35.820372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:60969 (local address 127.0.0.1:33193)
0810 18:47:35.821290 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:35.821303 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:35.821344 (+ 41us) server_negotiation.cc:407] Connection header received
0810 18:47:35.823126 (+ 1782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:35.823151 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:35.823487 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:35.823884 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:35.825019 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:35.825025 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:35.825042 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:47:35.825066 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:35.825169 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:47:35.825190 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:35.825450 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:35.825596 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 18:47:35.825788 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":411,"threads_started":1}
I20260810 18:47:35.831506 5011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:47:35.834676 5819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e3f5931bfd84108b43d542ace1cac75: Bootstrap starting.
I20260810 18:47:35.838793 5819 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e3f5931bfd84108b43d542ace1cac75: Neither blocks nor log segments found. Creating new log.
I20260810 18:47:35.839704 5823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:35.836553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.136:49793 (local address 127.0.0.1:33193)
0810 18:47:35.836742 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:35.836747 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:35.837307 (+ 560us) server_negotiation.cc:407] Connection header received
0810 18:47:35.837545 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:35.837558 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:35.837624 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:35.837746 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:35.838934 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:35.838940 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:35.838945 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:47:35.838971 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:35.839038 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:47:35.839057 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:35.839221 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:35.839311 (+ 90us) server_negotiation.cc:299] Negotiation successful
0810 18:47:35.839390 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 18:47:35.840204 5819 log.cc:824] T 00000000000000000000000000000000 P 3e3f5931bfd84108b43d542ace1cac75: Log is configured to *not* fsync() on all Append() calls
I20260810 18:47:35.843422 5819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e3f5931bfd84108b43d542ace1cac75: No bootstrap required, opened a new log
I20260810 18:47:35.851202 5819 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e3f5931bfd84108b43d542ace1cac75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e3f5931bfd84108b43d542ace1cac75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33193 } }
I20260810 18:47:35.851699 5819 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e3f5931bfd84108b43d542ace1cac75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:47:35.851852 5819 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e3f5931bfd84108b43d542ace1cac75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e3f5931bfd84108b43d542ace1cac75, State: Initialized, Role: FOLLOWER
I20260810 18:47:35.852623 5819 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e3f5931bfd84108b43d542ace1cac75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e3f5931bfd84108b43d542ace1cac75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33193 } }
I20260810 18:47:35.852857 5819 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e3f5931bfd84108b43d542ace1cac75 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:47:35.852977 5819 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e3f5931bfd84108b43d542ace1cac75 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:47:35.853130 5819 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e3f5931bfd84108b43d542ace1cac75 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:47:35.855584 5819 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e3f5931bfd84108b43d542ace1cac75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e3f5931bfd84108b43d542ace1cac75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33193 } }
I20260810 18:47:35.856180 5819 leader_election.cc:302] T 00000000000000000000000000000000 P 3e3f5931bfd84108b43d542ace1cac75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e3f5931bfd84108b43d542ace1cac75; no voters:
I20260810 18:47:35.856943 5819 leader_election.cc:288] T 00000000000000000000000000000000 P 3e3f5931bfd84108b43d542ace1cac75 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:47:35.857151 5833 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e3f5931bfd84108b43d542ace1cac75 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:47:35.858058 5833 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e3f5931bfd84108b43d542ace1cac75 [term 1 LEADER]: Becoming Leader. State: Replica: 3e3f5931bfd84108b43d542ace1cac75, State: Running, Role: LEADER
I20260810 18:47:35.858801 5833 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e3f5931bfd84108b43d542ace1cac75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e3f5931bfd84108b43d542ace1cac75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33193 } }
I20260810 18:47:35.859517 5819 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e3f5931bfd84108b43d542ace1cac75 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:47:35.863157 5835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e3f5931bfd84108b43d542ace1cac75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e3f5931bfd84108b43d542ace1cac75. Latest consensus state: current_term: 1 leader_uuid: "3e3f5931bfd84108b43d542ace1cac75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e3f5931bfd84108b43d542ace1cac75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33193 } } }
I20260810 18:47:35.863695 5835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e3f5931bfd84108b43d542ace1cac75 [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:35.862238 5834 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e3f5931bfd84108b43d542ace1cac75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e3f5931bfd84108b43d542ace1cac75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e3f5931bfd84108b43d542ace1cac75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33193 } } }
I20260810 18:47:35.865669 5834 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e3f5931bfd84108b43d542ace1cac75 [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:35.869020 5845 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:47:35.874814 5845 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:47:35.885550 5845 catalog_manager.cc:1347] Generated new cluster ID: a0267b700eec401e96128f8b4db58308
I20260810 18:47:35.885699 5845 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:47:35.919978 5845 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:47:35.920624 5845 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:47:35.929402 5845 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e3f5931bfd84108b43d542ace1cac75: Generated new TSK 0
I20260810 18:47:35.930493 5845 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:47:35.950027 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 5710
W20260810 18:47:35.951447 5812 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.136:60969
W20260810 18:47:35.952289 5011 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 5710: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:47:35.952474 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 5710
2026-08-10T18:47:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2380 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:47:36 dist-test-slave-7kzw krb5kdc[5862](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:47:36 dist-test-slave-7kzw krb5kdc[5862](info): set up 2 sockets
Aug 10 18:47:36 dist-test-slave-7kzw krb5kdc[5862](info): commencing operation
krb5kdc: starting...
W20260810 18:47:37.363292 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.362s user 0.001s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:47:37 dist-test-slave-7kzw krb5kdc[5862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:47:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:47:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.228.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
I20260810 18:47:37.743505 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:42221
--webserver_interface=127.4.228.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:38965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:42221
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:47:38.124958 5880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:47:38.125581 5880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:47:38.125816 5880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:47:38.134092 5880 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:47:38.134311 5880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:47:38.134423 5880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:47:38.134546 5880 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:47:38.134624 5880 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:47:38.145262 5880 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:38965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:42221
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:42221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:47:38.146883 5880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:47:38.148406 5880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:47:38.161571 5887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:38.161651 5886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:38.161653 5892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:47:38.162981 5880 server_base.cc:1034] running on GCE node
I20260810 18:47:38.163946 5880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:47:38.165544 5880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:47:38.166818 5880 hybrid_clock.cc:648] HybridClock initialized: now 1786387658166745 us; error 59 us; skew 500 ppm
I20260810 18:47:38.167207 5880 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:47:38 dist-test-slave-7kzw krb5kdc[5862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387658, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:47:38.170899 5880 init.cc:377] Logged in from keytab as kudu/127.4.228.254@KRBTEST.COM (short username kudu)
I20260810 18:47:38.172561 5880 webserver.cc:460] Webserver started at http://127.4.228.254:41399/ using document root <none> and password file <none>
I20260810 18:47:38.173327 5880 fs_manager.cc:359] Metadata directory not provided
I20260810 18:47:38.173465 5880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:47:38.173825 5880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:47:38.177175 5880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "c6adef7d406042a69605c115657e636b"
format_stamp: "Formatted at 2026-08-10 18:47:38 on dist-test-slave-7kzw"
server_key: "475f175ff0e8c97e6e6fbf4b5c6157a7"
server_key_iv: "aa73e628be9581fc2ce43c3836752199"
server_key_version: "encryptionkey@0"
I20260810 18:47:38.178151 5880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "c6adef7d406042a69605c115657e636b"
format_stamp: "Formatted at 2026-08-10 18:47:38 on dist-test-slave-7kzw"
server_key: "475f175ff0e8c97e6e6fbf4b5c6157a7"
server_key_iv: "aa73e628be9581fc2ce43c3836752199"
server_key_version: "encryptionkey@0"
I20260810 18:47:38.184716 5880 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260810 18:47:38.189685 5903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:38.191512 5880 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260810 18:47:38.191788 5880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "c6adef7d406042a69605c115657e636b"
format_stamp: "Formatted at 2026-08-10 18:47:38 on dist-test-slave-7kzw"
server_key: "475f175ff0e8c97e6e6fbf4b5c6157a7"
server_key_iv: "aa73e628be9581fc2ce43c3836752199"
server_key_version: "encryptionkey@0"
I20260810 18:47:38.192029 5880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:47:38.211124 5880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:47:38.216246 5880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:47:38.216575 5880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:47:38.238219 5880 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:42221
I20260810 18:47:38.238240 5978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:42221 every 8 connection(s)
I20260810 18:47:38.239892 5880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:47:38.241338 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 5880
I20260810 18:47:38.241544 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:47:38.242151 5011 external_mini_cluster.cc:1428] Setting key 6d753d75dac2e35444459561764b7d8d
I20260810 18:47:38.245080 5979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 18:47:38 dist-test-slave-7kzw krb5kdc[5862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:47:38.260308 5979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6adef7d406042a69605c115657e636b: Bootstrap starting.
I20260810 18:47:38.263878 5979 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6adef7d406042a69605c115657e636b: Neither blocks nor log segments found. Creating new log.
I20260810 18:47:38.264964 5979 log.cc:824] T 00000000000000000000000000000000 P c6adef7d406042a69605c115657e636b: Log is configured to *not* fsync() on all Append() calls
I20260810 18:47:38.268630 5979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6adef7d406042a69605c115657e636b: No bootstrap required, opened a new log
I20260810 18:47:38.269980 5984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:38.244209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55365 (local address 127.4.228.254:42221)
0810 18:47:38.245088 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:38.245100 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:38.245170 (+ 70us) server_negotiation.cc:407] Connection header received
0810 18:47:38.246960 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:38.246984 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:38.247404 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:38.247772 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:38.249696 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:38.251266 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:38.253502 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:38.254312 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:38.254558 (+ 246us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:38.259967 (+ 5409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:38.259977 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:38.259995 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:47:38.260046 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:38.263933 (+ 3887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:38.264980 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:38.264989 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:38.264997 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:38.265389 (+ 392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:38.266018 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:38.266023 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:38.266025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:38.266819 (+ 794us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:38.267197 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:38.267877 (+ 680us) server_negotiation.cc:299] Negotiation successful
0810 18:47:38.268559 (+ 682us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":406,"threads_started":1}
I20260810 18:47:38.274657 5011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:47:38.278956 5979 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6adef7d406042a69605c115657e636b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6adef7d406042a69605c115657e636b" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42221 } }
I20260810 18:47:38.279696 5979 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6adef7d406042a69605c115657e636b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:47:38.279842 5979 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6adef7d406042a69605c115657e636b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6adef7d406042a69605c115657e636b, State: Initialized, Role: FOLLOWER
I20260810 18:47:38.280602 5979 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6adef7d406042a69605c115657e636b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6adef7d406042a69605c115657e636b" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42221 } }
I20260810 18:47:38.280952 5979 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6adef7d406042a69605c115657e636b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:47:38.281131 5979 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6adef7d406042a69605c115657e636b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:47:38.281350 5979 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6adef7d406042a69605c115657e636b [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:47:38.284698 5979 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6adef7d406042a69605c115657e636b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6adef7d406042a69605c115657e636b" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42221 } }
I20260810 18:47:38.285393 5979 leader_election.cc:302] T 00000000000000000000000000000000 P c6adef7d406042a69605c115657e636b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6adef7d406042a69605c115657e636b; no voters:
I20260810 18:47:38.286136 5979 leader_election.cc:288] T 00000000000000000000000000000000 P c6adef7d406042a69605c115657e636b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:47:38.286291 5997 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6adef7d406042a69605c115657e636b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:47:38.288843 5979 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6adef7d406042a69605c115657e636b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:47:38.288981 5997 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6adef7d406042a69605c115657e636b [term 1 LEADER]: Becoming Leader. State: Replica: c6adef7d406042a69605c115657e636b, State: Running, Role: LEADER
I20260810 18:47:38.289819 5997 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6adef7d406042a69605c115657e636b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6adef7d406042a69605c115657e636b" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42221 } }
I20260810 18:47:38.293757 6001 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6adef7d406042a69605c115657e636b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6adef7d406042a69605c115657e636b. Latest consensus state: current_term: 1 leader_uuid: "c6adef7d406042a69605c115657e636b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6adef7d406042a69605c115657e636b" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42221 } } }
I20260810 18:47:38.294327 6001 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6adef7d406042a69605c115657e636b [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:38.293788 5998 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6adef7d406042a69605c115657e636b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c6adef7d406042a69605c115657e636b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6adef7d406042a69605c115657e636b" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42221 } } }
I20260810 18:47:38.294497 5998 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6adef7d406042a69605c115657e636b [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:38.298501 6010 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:47:38.300731 5984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:38.280655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36477 (local address 127.4.228.254:42221)
0810 18:47:38.280890 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:38.280897 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:38.281528 (+ 631us) server_negotiation.cc:407] Connection header received
0810 18:47:38.281847 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:38.281852 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:38.281919 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:38.282006 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:38.283955 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:38.284742 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:38.286868 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:38.287582 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:38.287694 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:38.291524 (+ 3830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:38.291532 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:38.291536 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:47:38.291582 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:38.296107 (+ 4525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:38.297687 (+ 1580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:38.297692 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:38.297695 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:38.297929 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:38.298572 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:38.298577 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:38.298583 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:38.298946 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:38.299446 (+ 500us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:38.300098 (+ 652us) server_negotiation.cc:299] Negotiation successful
0810 18:47:38.300317 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 18:47:38.306552 6010 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:47:38.316521 6010 catalog_manager.cc:1347] Generated new cluster ID: 37a9fb0b9c44441e978423e824bab80a
I20260810 18:47:38.316665 6010 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:47:38.340265 6010 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:47:38.341996 6010 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:47:38.348096 6010 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6adef7d406042a69605c115657e636b: Generated new TSK 0
I20260810 18:47:38.349092 6010 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:47:38.358877 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 5880
I20260810 18:47:38.360661 5972 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:47:38.456650 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 5880
W20260810 18:47:38.484733 5860 connection.cc:535] client connection to 127.4.228.254:42221 recv error: Network error: failed to read from TLS socket (remote: 127.4.228.254:42221): Connection reset by peer (error 104)
2026-08-10T18:47:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2503 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:47:38 dist-test-slave-7kzw krb5kdc[6024](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:47:38 dist-test-slave-7kzw krb5kdc[6024](info): set up 2 sockets
Aug 10 18:47:38 dist-test-slave-7kzw krb5kdc[6024](info): commencing operation
krb5kdc: starting...
W20260810 18:47:39.864094 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.359s user 0.001s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:47:40 dist-test-slave-7kzw krb5kdc[6024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:47:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:47:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.228.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
I20260810 18:47:40.235196 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:41137
--webserver_interface=127.4.228.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:44767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:41137
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:47:40.609994 6040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:47:40.610324 6040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:47:40.610427 6040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:47:40.618604 6040 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:47:40.618754 6040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:47:40.618803 6040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:47:40.618860 6040 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:47:40.618906 6040 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:47:40.629590 6040 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:44767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:41137
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:41137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:47:40.631017 6040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:47:40.632510 6040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:47:40.645665 6047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:40.645737 6055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:40.645697 6048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:47:40.646961 6040 server_base.cc:1034] running on GCE node
I20260810 18:47:40.647944 6040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:47:40.649503 6040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:47:40.650787 6040 hybrid_clock.cc:648] HybridClock initialized: now 1786387660650707 us; error 70 us; skew 500 ppm
I20260810 18:47:40.651278 6040 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:47:40 dist-test-slave-7kzw krb5kdc[6024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387660, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:47:40.655328 6040 init.cc:377] Logged in from keytab as kudu/127.4.228.254@KRBTEST.COM (short username kudu)
I20260810 18:47:40.656903 6040 webserver.cc:460] Webserver started at http://127.4.228.254:36503/ using document root <none> and password file <none>
I20260810 18:47:40.657598 6040 fs_manager.cc:359] Metadata directory not provided
I20260810 18:47:40.657716 6040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:47:40.658097 6040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:47:40.661826 6040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "080c46c11544462e8e8a05df980dc6df"
format_stamp: "Formatted at 2026-08-10 18:47:40 on dist-test-slave-7kzw"
server_key: "81675559102659e91d0f2347914301ba"
server_key_iv: "b7a41e7dcaf813add26101ce3bf75af3"
server_key_version: "encryptionkey@0"
I20260810 18:47:40.662693 6040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "080c46c11544462e8e8a05df980dc6df"
format_stamp: "Formatted at 2026-08-10 18:47:40 on dist-test-slave-7kzw"
server_key: "81675559102659e91d0f2347914301ba"
server_key_iv: "b7a41e7dcaf813add26101ce3bf75af3"
server_key_version: "encryptionkey@0"
I20260810 18:47:40.668371 6040 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 18:47:40.672457 6063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:40.674108 6040 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260810 18:47:40.674337 6040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "080c46c11544462e8e8a05df980dc6df"
format_stamp: "Formatted at 2026-08-10 18:47:40 on dist-test-slave-7kzw"
server_key: "81675559102659e91d0f2347914301ba"
server_key_iv: "b7a41e7dcaf813add26101ce3bf75af3"
server_key_version: "encryptionkey@0"
I20260810 18:47:40.674644 6040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:47:40.709832 6040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:47:40.715106 6040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:47:40.715487 6040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:47:40.737211 6040 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:41137
I20260810 18:47:40.737273 6138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:41137 every 8 connection(s)
I20260810 18:47:40.739282 6040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:47:40.743918 6144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:47:40.744053 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 6040
I20260810 18:47:40.744247 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:47:40.744856 5011 external_mini_cluster.cc:1428] Setting key ab4d7f733a0c73c33725096dbb692b90
I20260810 18:47:40.759569 6144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 080c46c11544462e8e8a05df980dc6df: Bootstrap starting.
I20260810 18:47:40.761063 6150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:40.746939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42733 (local address 127.4.228.254:41137)
0810 18:47:40.747965 (+ 1026us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:40.747977 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:40.748024 (+ 47us) server_negotiation.cc:407] Connection header received
0810 18:47:40.750561 (+ 2537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:40.750604 (+ 43us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:40.751053 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:40.751444 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:40.752371 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:40.753734 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:40.755961 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:40.756778 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:40.756934 (+ 156us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:40.758414 (+ 1480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:40.758422 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:40.758438 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:47:40.759141 (+ 703us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:40.759238 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:47:40.759266 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:40.759395 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:40.759842 (+ 447us) server_negotiation.cc:299] Negotiation successful
0810 18:47:40.760008 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":781,"thread_start_us":509,"threads_started":1}
I20260810 18:47:40.763480 6144 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 080c46c11544462e8e8a05df980dc6df: Neither blocks nor log segments found. Creating new log.
I20260810 18:47:40.764981 6144 log.cc:824] T 00000000000000000000000000000000 P 080c46c11544462e8e8a05df980dc6df: Log is configured to *not* fsync() on all Append() calls
I20260810 18:47:40.765450 5011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:47:40.768878 6144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 080c46c11544462e8e8a05df980dc6df: No bootstrap required, opened a new log
I20260810 18:47:40.779546 6150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:40.771673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45317 (local address 127.4.228.254:41137)
0810 18:47:40.771892 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:40.771899 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:40.772349 (+ 450us) server_negotiation.cc:407] Connection header received
0810 18:47:40.772534 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:40.772541 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:40.772643 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:40.772765 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:40.773328 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:40.774554 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:40.776070 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:40.776788 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:40.776951 (+ 163us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:40.778294 (+ 1343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:40.778305 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:40.778313 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:47:40.778337 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:40.778389 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:47:40.778410 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:40.778581 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:40.779125 (+ 544us) server_negotiation.cc:299] Negotiation successful
0810 18:47:40.779188 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 18:47:40.779585 6144 raft_consensus.cc:348] T 00000000000000000000000000000000 P 080c46c11544462e8e8a05df980dc6df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "080c46c11544462e8e8a05df980dc6df" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 41137 } }
I20260810 18:47:40.780288 6144 raft_consensus.cc:374] T 00000000000000000000000000000000 P 080c46c11544462e8e8a05df980dc6df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:47:40.780447 6144 raft_consensus.cc:729] T 00000000000000000000000000000000 P 080c46c11544462e8e8a05df980dc6df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 080c46c11544462e8e8a05df980dc6df, State: Initialized, Role: FOLLOWER
I20260810 18:47:40.781221 6144 consensus_queue.cc:260] T 00000000000000000000000000000000 P 080c46c11544462e8e8a05df980dc6df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "080c46c11544462e8e8a05df980dc6df" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 41137 } }
I20260810 18:47:40.781594 6144 raft_consensus.cc:388] T 00000000000000000000000000000000 P 080c46c11544462e8e8a05df980dc6df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:47:40.781780 6144 raft_consensus.cc:482] T 00000000000000000000000000000000 P 080c46c11544462e8e8a05df980dc6df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:47:40.781982 6144 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 080c46c11544462e8e8a05df980dc6df [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:47:40.784400 6144 raft_consensus.cc:504] T 00000000000000000000000000000000 P 080c46c11544462e8e8a05df980dc6df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "080c46c11544462e8e8a05df980dc6df" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 41137 } }
I20260810 18:47:40.785004 6144 leader_election.cc:302] T 00000000000000000000000000000000 P 080c46c11544462e8e8a05df980dc6df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 080c46c11544462e8e8a05df980dc6df; no voters:
I20260810 18:47:40.785641 6144 leader_election.cc:288] T 00000000000000000000000000000000 P 080c46c11544462e8e8a05df980dc6df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:47:40.785827 6158 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 080c46c11544462e8e8a05df980dc6df [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:47:40.786682 6158 raft_consensus.cc:686] T 00000000000000000000000000000000 P 080c46c11544462e8e8a05df980dc6df [term 1 LEADER]: Becoming Leader. State: Replica: 080c46c11544462e8e8a05df980dc6df, State: Running, Role: LEADER
I20260810 18:47:40.787447 6158 consensus_queue.cc:237] T 00000000000000000000000000000000 P 080c46c11544462e8e8a05df980dc6df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "080c46c11544462e8e8a05df980dc6df" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 41137 } }
I20260810 18:47:40.788321 6144 sys_catalog.cc:564] T 00000000000000000000000000000000 P 080c46c11544462e8e8a05df980dc6df [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:47:40.791083 6160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 080c46c11544462e8e8a05df980dc6df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "080c46c11544462e8e8a05df980dc6df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "080c46c11544462e8e8a05df980dc6df" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 41137 } } }
I20260810 18:47:40.791489 6160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 080c46c11544462e8e8a05df980dc6df [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:40.791049 6161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 080c46c11544462e8e8a05df980dc6df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 080c46c11544462e8e8a05df980dc6df. Latest consensus state: current_term: 1 leader_uuid: "080c46c11544462e8e8a05df980dc6df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "080c46c11544462e8e8a05df980dc6df" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 41137 } } }
I20260810 18:47:40.792121 6161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 080c46c11544462e8e8a05df980dc6df [sys.catalog]: This master's current role is: LEADER
W20260810 18:47:40.804808 6183 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 080c46c11544462e8e8a05df980dc6df: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:47:40.804992 6183 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:47:40.805194 6169 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:47:40.807644 6169 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:47:40.816941 6169 catalog_manager.cc:1347] Generated new cluster ID: 9c2f49d1b22746a997ae2c4890c89f66
I20260810 18:47:40.817085 6169 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:47:40.833004 6169 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:47:40.834952 6169 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:47:40.843456 6169 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 080c46c11544462e8e8a05df980dc6df: Generated new TSK 0
I20260810 18:47:40.844692 6169 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:47:40.855185 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 6040
W20260810 18:47:40.856488 6131 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:42733
W20260810 18:47:40.857445 5011 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 6040: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:47:40.857604 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 6040
W20260810 18:47:40.887699 6022 connection.cc:535] client connection to 127.4.228.254:41137 recv error: Network error: failed to read from TLS socket (remote: 127.4.228.254:41137): Connection reset by peer (error 104)
2026-08-10T18:47:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2402 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:47:41 dist-test-slave-7kzw krb5kdc[6191](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:47:41 dist-test-slave-7kzw krb5kdc[6191](info): set up 2 sockets
Aug 10 18:47:41 dist-test-slave-7kzw krb5kdc[6191](info): commencing operation
krb5kdc: starting...
W20260810 18:47:42.250432 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.342s user 0.000s sys 0.055s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:47:42 dist-test-slave-7kzw krb5kdc[6191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:47:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:47:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.228.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
I20260810 18:47:42.620985 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:46189
--webserver_interface=127.4.228.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:34121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:46189
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:47:42.996028 6209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:47:42.996351 6209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:47:42.996524 6209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:47:43.004729 6209 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:47:43.004882 6209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:47:43.004966 6209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:47:43.005087 6209 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:47:43.005151 6209 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:47:43.016270 6209 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:34121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:46189
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:46189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:47:43.017768 6209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:47:43.019345 6209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:47:43.032596 6219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:43.032752 6214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:43.032724 6216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:47:43.033800 6209 server_base.cc:1034] running on GCE node
I20260810 18:47:43.034909 6209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:47:43.036559 6209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:47:43.037863 6209 hybrid_clock.cc:648] HybridClock initialized: now 1786387663037798 us; error 56 us; skew 500 ppm
I20260810 18:47:43.038331 6209 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:47:43 dist-test-slave-7kzw krb5kdc[6191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387663, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:47:43.042089 6209 init.cc:377] Logged in from keytab as kudu/127.4.228.254@KRBTEST.COM (short username kudu)
I20260810 18:47:43.043959 6209 webserver.cc:460] Webserver started at http://127.4.228.254:38153/ using document root <none> and password file <none>
I20260810 18:47:43.044734 6209 fs_manager.cc:359] Metadata directory not provided
I20260810 18:47:43.044895 6209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:47:43.045321 6209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:47:43.048666 6209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "97810cdc749046e9bc636af0553edfa2"
format_stamp: "Formatted at 2026-08-10 18:47:43 on dist-test-slave-7kzw"
server_key: "05a371d876d9f862c697a15dacb56c2f"
server_key_iv: "86ac37d603bd1183b98e87b0c61c11e7"
server_key_version: "encryptionkey@0"
I20260810 18:47:43.049506 6209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "97810cdc749046e9bc636af0553edfa2"
format_stamp: "Formatted at 2026-08-10 18:47:43 on dist-test-slave-7kzw"
server_key: "05a371d876d9f862c697a15dacb56c2f"
server_key_iv: "86ac37d603bd1183b98e87b0c61c11e7"
server_key_version: "encryptionkey@0"
I20260810 18:47:43.055384 6209 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 18:47:43.059648 6234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:43.061122 6209 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260810 18:47:43.061352 6209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "97810cdc749046e9bc636af0553edfa2"
format_stamp: "Formatted at 2026-08-10 18:47:43 on dist-test-slave-7kzw"
server_key: "05a371d876d9f862c697a15dacb56c2f"
server_key_iv: "86ac37d603bd1183b98e87b0c61c11e7"
server_key_version: "encryptionkey@0"
I20260810 18:47:43.061576 6209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:47:43.068078 6209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:47:43.073179 6209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:47:43.073529 6209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:47:43.094215 6209 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:46189
I20260810 18:47:43.094264 6310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:46189 every 8 connection(s)
I20260810 18:47:43.095813 6209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:47:43.097596 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 6209
I20260810 18:47:43.097813 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:47:43.098492 5011 external_mini_cluster.cc:1428] Setting key 2f895bf25cf3d248ecbd8b77869f4605
I20260810 18:47:43.101377 6311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:47:43.107034 6316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:43.100539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49813 (local address 127.4.228.254:46189)
0810 18:47:43.101492 (+ 953us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:43.101512 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:43.101573 (+ 61us) server_negotiation.cc:407] Connection header received
0810 18:47:43.103356 (+ 1783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:43.103380 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:43.103711 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:43.104124 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:43.105234 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:43.105240 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:43.105261 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:47:43.105277 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:43.105360 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:47:43.105380 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:43.105645 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:43.105791 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 18:47:43.105989 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":394,"threads_started":1}
I20260810 18:47:43.111063 5011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:47:43.114569 6311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97810cdc749046e9bc636af0553edfa2: Bootstrap starting.
I20260810 18:47:43.118762 6311 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97810cdc749046e9bc636af0553edfa2: Neither blocks nor log segments found. Creating new log.
I20260810 18:47:43.119097 6316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:43.115969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41831 (local address 127.4.228.254:46189)
0810 18:47:43.116173 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:43.116181 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:43.116783 (+ 602us) server_negotiation.cc:407] Connection header received
0810 18:47:43.116932 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:43.116938 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:43.117003 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:43.117122 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:43.118256 (+ 1134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:43.118261 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:43.118266 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:47:43.118287 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:43.118335 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:47:43.118356 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:43.118500 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:43.118731 (+ 231us) server_negotiation.cc:299] Negotiation successful
0810 18:47:43.118801 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 18:47:43.120085 6311 log.cc:824] T 00000000000000000000000000000000 P 97810cdc749046e9bc636af0553edfa2: Log is configured to *not* fsync() on all Append() calls
I20260810 18:47:43.123104 6311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97810cdc749046e9bc636af0553edfa2: No bootstrap required, opened a new log
I20260810 18:47:43.131276 6311 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97810cdc749046e9bc636af0553edfa2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97810cdc749046e9bc636af0553edfa2" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46189 } }
I20260810 18:47:43.131754 6311 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97810cdc749046e9bc636af0553edfa2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:47:43.131870 6311 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97810cdc749046e9bc636af0553edfa2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97810cdc749046e9bc636af0553edfa2, State: Initialized, Role: FOLLOWER
I20260810 18:47:43.132587 6311 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97810cdc749046e9bc636af0553edfa2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97810cdc749046e9bc636af0553edfa2" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46189 } }
I20260810 18:47:43.132805 6311 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97810cdc749046e9bc636af0553edfa2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:47:43.132994 6311 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97810cdc749046e9bc636af0553edfa2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:47:43.133201 6311 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97810cdc749046e9bc636af0553edfa2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:47:43.135576 6311 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97810cdc749046e9bc636af0553edfa2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97810cdc749046e9bc636af0553edfa2" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46189 } }
I20260810 18:47:43.136173 6311 leader_election.cc:302] T 00000000000000000000000000000000 P 97810cdc749046e9bc636af0553edfa2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97810cdc749046e9bc636af0553edfa2; no voters:
I20260810 18:47:43.136871 6311 leader_election.cc:288] T 00000000000000000000000000000000 P 97810cdc749046e9bc636af0553edfa2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:47:43.137051 6326 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97810cdc749046e9bc636af0553edfa2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:47:43.137770 6326 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97810cdc749046e9bc636af0553edfa2 [term 1 LEADER]: Becoming Leader. State: Replica: 97810cdc749046e9bc636af0553edfa2, State: Running, Role: LEADER
I20260810 18:47:43.138419 6326 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97810cdc749046e9bc636af0553edfa2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97810cdc749046e9bc636af0553edfa2" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46189 } }
I20260810 18:47:43.139441 6311 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97810cdc749046e9bc636af0553edfa2 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:47:43.141881 6328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97810cdc749046e9bc636af0553edfa2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97810cdc749046e9bc636af0553edfa2. Latest consensus state: current_term: 1 leader_uuid: "97810cdc749046e9bc636af0553edfa2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97810cdc749046e9bc636af0553edfa2" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46189 } } }
I20260810 18:47:43.142242 6327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97810cdc749046e9bc636af0553edfa2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97810cdc749046e9bc636af0553edfa2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97810cdc749046e9bc636af0553edfa2" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46189 } } }
I20260810 18:47:43.142411 6328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97810cdc749046e9bc636af0553edfa2 [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:43.142614 6327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97810cdc749046e9bc636af0553edfa2 [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:43.144408 6336 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:47:43.150157 6336 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:47:43.160156 6336 catalog_manager.cc:1347] Generated new cluster ID: e2a4b0d9635743eea3a96ebdc227a634
I20260810 18:47:43.160310 6336 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:47:43.185420 6336 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:47:43.186110 6336 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:47:43.204725 6336 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97810cdc749046e9bc636af0553edfa2: Generated new TSK 0
I20260810 18:47:43.206019 6336 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:47:43.216451 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 6209
W20260810 18:47:43.218019 6305 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:49813
W20260810 18:47:43.218933 5011 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 6209: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:47:43.219115 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 6209
2026-08-10T18:47:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2360 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:47:43 dist-test-slave-7kzw krb5kdc[6350](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:47:43 dist-test-slave-7kzw krb5kdc[6350](info): set up 2 sockets
Aug 10 18:47:43 dist-test-slave-7kzw krb5kdc[6350](info): commencing operation
krb5kdc: starting...
W20260810 18:47:44.655951 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.388s user 0.001s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:47:44 dist-test-slave-7kzw krb5kdc[6350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:47:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:47:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.228.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
I20260810 18:47:45.056301 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:44593
--webserver_interface=127.4.228.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:37959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:44593
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:47:45.449417 6372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:47:45.450055 6372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:47:45.450378 6372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:47:45.458776 6372 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:47:45.458943 6372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:47:45.459040 6372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:47:45.459136 6372 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:47:45.459225 6372 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:47:45.470233 6372 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:37959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:44593
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:44593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:47:45.472520 6372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:47:45.474424 6372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:47:45.488986 6382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:45.488957 6385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:45.489382 6383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:47:45.489766 6372 server_base.cc:1034] running on GCE node
I20260810 18:47:45.491283 6372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:47:45.493146 6372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:47:45.494590 6372 hybrid_clock.cc:648] HybridClock initialized: now 1786387665494560 us; error 82 us; skew 500 ppm
I20260810 18:47:45.495054 6372 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:47:45 dist-test-slave-7kzw krb5kdc[6350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387665, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:47:45.499356 6372 init.cc:377] Logged in from keytab as kudu/127.4.228.254@KRBTEST.COM (short username kudu)
I20260810 18:47:45.501214 6372 webserver.cc:460] Webserver started at http://127.4.228.254:42167/ using document root <none> and password file <none>
I20260810 18:47:45.502055 6372 fs_manager.cc:359] Metadata directory not provided
I20260810 18:47:45.502228 6372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:47:45.502672 6372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:47:45.506103 6372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "8990f6a278d449b8bb72d7c7ab873b91"
format_stamp: "Formatted at 2026-08-10 18:47:45 on dist-test-slave-7kzw"
server_key: "d75275f562b4a9206808f5119d446a1f"
server_key_iv: "95b9846cf3f75b429d0d336d537e5d2e"
server_key_version: "encryptionkey@0"
I20260810 18:47:45.507042 6372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "8990f6a278d449b8bb72d7c7ab873b91"
format_stamp: "Formatted at 2026-08-10 18:47:45 on dist-test-slave-7kzw"
server_key: "d75275f562b4a9206808f5119d446a1f"
server_key_iv: "95b9846cf3f75b429d0d336d537e5d2e"
server_key_version: "encryptionkey@0"
I20260810 18:47:45.513978 6372 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260810 18:47:45.518440 6392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:45.520130 6372 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 18:47:45.520406 6372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "8990f6a278d449b8bb72d7c7ab873b91"
format_stamp: "Formatted at 2026-08-10 18:47:45 on dist-test-slave-7kzw"
server_key: "d75275f562b4a9206808f5119d446a1f"
server_key_iv: "95b9846cf3f75b429d0d336d537e5d2e"
server_key_version: "encryptionkey@0"
I20260810 18:47:45.520653 6372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:47:45.544756 6372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:47:45.550499 6372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:47:45.550928 6372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:47:45.573284 6372 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:44593
I20260810 18:47:45.573288 6461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:44593 every 8 connection(s)
I20260810 18:47:45.575047 6372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:47:45.578938 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 6372
I20260810 18:47:45.579138 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:47:45.579741 5011 external_mini_cluster.cc:1428] Setting key fd785fdf489e830a4222df3bb76e4035
I20260810 18:47:45.579813 6462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:47:45.597695 6462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8990f6a278d449b8bb72d7c7ab873b91: Bootstrap starting.
Aug 10 18:47:45 dist-test-slave-7kzw krb5kdc[6350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:47:45.601759 6462 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8990f6a278d449b8bb72d7c7ab873b91: Neither blocks nor log segments found. Creating new log.
I20260810 18:47:45.603037 6462 log.cc:824] T 00000000000000000000000000000000 P 8990f6a278d449b8bb72d7c7ab873b91: Log is configured to *not* fsync() on all Append() calls
I20260810 18:47:45.608265 6462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8990f6a278d449b8bb72d7c7ab873b91: No bootstrap required, opened a new log
I20260810 18:47:45.608959 6467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:45.581756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46847 (local address 127.4.228.254:44593)
0810 18:47:45.582880 (+ 1124us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:45.582893 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:45.582941 (+ 48us) server_negotiation.cc:407] Connection header received
0810 18:47:45.585848 (+ 2907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:45.585878 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:45.586370 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:45.586853 (+ 483us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:45.589129 (+ 2276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:45.590605 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:45.592702 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:45.593749 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:45.593925 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:45.599625 (+ 5700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:45.599635 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:45.599652 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:47:45.599714 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:45.603283 (+ 3569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:45.604228 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:45.604240 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:45.604255 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:45.604493 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:45.605301 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:45.605305 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:45.605307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:45.605898 (+ 591us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:45.606147 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:45.607234 (+ 1087us) server_negotiation.cc:299] Negotiation successful
0810 18:47:45.607624 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":864,"thread_start_us":614,"threads_started":1}
I20260810 18:47:45.613261 5011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:47:45.617043 6462 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8990f6a278d449b8bb72d7c7ab873b91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8990f6a278d449b8bb72d7c7ab873b91" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 44593 } }
I20260810 18:47:45.617473 6462 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8990f6a278d449b8bb72d7c7ab873b91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:47:45.617622 6462 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8990f6a278d449b8bb72d7c7ab873b91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8990f6a278d449b8bb72d7c7ab873b91, State: Initialized, Role: FOLLOWER
I20260810 18:47:45.618348 6462 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8990f6a278d449b8bb72d7c7ab873b91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8990f6a278d449b8bb72d7c7ab873b91" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 44593 } }
I20260810 18:47:45.618726 6462 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8990f6a278d449b8bb72d7c7ab873b91 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:47:45.618880 6462 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8990f6a278d449b8bb72d7c7ab873b91 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:47:45.619097 6462 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8990f6a278d449b8bb72d7c7ab873b91 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:47:45.621892 6462 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8990f6a278d449b8bb72d7c7ab873b91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8990f6a278d449b8bb72d7c7ab873b91" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 44593 } }
I20260810 18:47:45.622570 6462 leader_election.cc:302] T 00000000000000000000000000000000 P 8990f6a278d449b8bb72d7c7ab873b91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8990f6a278d449b8bb72d7c7ab873b91; no voters:
I20260810 18:47:45.623498 6462 leader_election.cc:288] T 00000000000000000000000000000000 P 8990f6a278d449b8bb72d7c7ab873b91 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:47:45.623732 6476 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8990f6a278d449b8bb72d7c7ab873b91 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:47:45.627592 6476 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8990f6a278d449b8bb72d7c7ab873b91 [term 1 LEADER]: Becoming Leader. State: Replica: 8990f6a278d449b8bb72d7c7ab873b91, State: Running, Role: LEADER
I20260810 18:47:45.628355 6476 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8990f6a278d449b8bb72d7c7ab873b91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8990f6a278d449b8bb72d7c7ab873b91" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 44593 } }
I20260810 18:47:45.628750 6462 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8990f6a278d449b8bb72d7c7ab873b91 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:47:45.632149 6477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8990f6a278d449b8bb72d7c7ab873b91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8990f6a278d449b8bb72d7c7ab873b91. Latest consensus state: current_term: 1 leader_uuid: "8990f6a278d449b8bb72d7c7ab873b91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8990f6a278d449b8bb72d7c7ab873b91" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 44593 } } }
I20260810 18:47:45.632671 6477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8990f6a278d449b8bb72d7c7ab873b91 [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:45.633759 6482 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:47:45.633597 6478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8990f6a278d449b8bb72d7c7ab873b91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8990f6a278d449b8bb72d7c7ab873b91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8990f6a278d449b8bb72d7c7ab873b91" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 44593 } } }
I20260810 18:47:45.635473 6478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8990f6a278d449b8bb72d7c7ab873b91 [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:45.641094 6467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:45.618202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35731 (local address 127.4.228.254:44593)
0810 18:47:45.618424 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:45.618431 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:45.618998 (+ 567us) server_negotiation.cc:407] Connection header received
0810 18:47:45.619234 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:45.619240 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:45.619333 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:45.619474 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:45.620986 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:45.622084 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:45.624506 (+ 2422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:45.625769 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:45.625952 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:45.630820 (+ 4868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:45.630828 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:45.630833 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:47:45.630880 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:45.634848 (+ 3968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:45.635800 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:45.635810 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:45.635816 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:45.636016 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:45.636585 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:45.636594 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:45.636600 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:45.637378 (+ 778us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:45.637756 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:45.640047 (+ 2291us) server_negotiation.cc:299] Negotiation successful
0810 18:47:45.640533 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 18:47:45.646544 6482 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:47:45.658700 6482 catalog_manager.cc:1347] Generated new cluster ID: 44cc59cc9a3941d3a66fa510db09d2f1
I20260810 18:47:45.658843 6482 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:47:45.682904 6482 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:47:45.684320 6482 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:47:45.693637 6482 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8990f6a278d449b8bb72d7c7ab873b91: Generated new TSK 0
I20260810 18:47:45.694885 6482 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:47:45.705058 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 6372
I20260810 18:47:45.706380 6456 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:47:45.806053 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 6372
2026-08-10T18:47:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2585 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:47:45 dist-test-slave-7kzw krb5kdc[6507](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:47:45 dist-test-slave-7kzw krb5kdc[6507](info): set up 2 sockets
Aug 10 18:47:45 dist-test-slave-7kzw krb5kdc[6507](info): commencing operation
krb5kdc: starting...
W20260810 18:47:47.219326 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.366s user 0.002s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:47:47 dist-test-slave-7kzw krb5kdc[6507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:47:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:47:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.228.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
I20260810 18:47:47.583688 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:33731
--webserver_interface=127.4.228.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:45515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:33731
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:47:47.985049 6530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:47:47.985498 6530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:47:47.985687 6530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:47:47.993378 6530 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:47:47.993558 6530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:47:47.993671 6530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:47:47.993800 6530 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:47:47.993883 6530 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:47:48.004434 6530 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:45515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:33731
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:33731
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:47:48.005950 6530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:47:48.007468 6530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:47:48.021317 6536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:48.021291 6540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:48.021368 6537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:47:48.023090 6530 server_base.cc:1034] running on GCE node
I20260810 18:47:48.024124 6530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:47:48.026341 6530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:47:48.027922 6530 hybrid_clock.cc:648] HybridClock initialized: now 1786387668027855 us; error 96 us; skew 500 ppm
I20260810 18:47:48.028901 6530 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:47:48 dist-test-slave-7kzw krb5kdc[6507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387668, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:47:48.033396 6530 init.cc:377] Logged in from keytab as kudu/127.4.228.254@KRBTEST.COM (short username kudu)
I20260810 18:47:48.035902 6530 webserver.cc:460] Webserver started at http://127.4.228.254:33139/ using document root <none> and password file <none>
I20260810 18:47:48.037106 6530 fs_manager.cc:359] Metadata directory not provided
I20260810 18:47:48.037320 6530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:47:48.037699 6530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:47:48.041143 6530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "2144bf89790344648965aaaf21e26928"
format_stamp: "Formatted at 2026-08-10 18:47:48 on dist-test-slave-7kzw"
server_key: "5357809f5a7b5a8752c5423799ef75f5"
server_key_iv: "226c90eb77f1fedfe24d419ed9f32b86"
server_key_version: "encryptionkey@0"
I20260810 18:47:48.042001 6530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "2144bf89790344648965aaaf21e26928"
format_stamp: "Formatted at 2026-08-10 18:47:48 on dist-test-slave-7kzw"
server_key: "5357809f5a7b5a8752c5423799ef75f5"
server_key_iv: "226c90eb77f1fedfe24d419ed9f32b86"
server_key_version: "encryptionkey@0"
I20260810 18:47:48.047902 6530 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 18:47:48.052493 6552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:48.054291 6530 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260810 18:47:48.054574 6530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "2144bf89790344648965aaaf21e26928"
format_stamp: "Formatted at 2026-08-10 18:47:48 on dist-test-slave-7kzw"
server_key: "5357809f5a7b5a8752c5423799ef75f5"
server_key_iv: "226c90eb77f1fedfe24d419ed9f32b86"
server_key_version: "encryptionkey@0"
I20260810 18:47:48.054847 6530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:47:48.073504 6530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:47:48.078172 6530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:47:48.078552 6530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:47:48.100152 6530 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:33731
I20260810 18:47:48.100203 6628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:33731 every 8 connection(s)
I20260810 18:47:48.101723 6530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:47:48.105916 6630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:47:48.112123 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 6530
I20260810 18:47:48.112367 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:47:48.112995 5011 external_mini_cluster.cc:1428] Setting key 797daab5705170ad78ef681db3c55fdf
I20260810 18:47:48.119546 6630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2144bf89790344648965aaaf21e26928: Bootstrap starting.
I20260810 18:47:48.123469 6630 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2144bf89790344648965aaaf21e26928: Neither blocks nor log segments found. Creating new log.
I20260810 18:47:48.124650 6630 log.cc:824] T 00000000000000000000000000000000 P 2144bf89790344648965aaaf21e26928: Log is configured to *not* fsync() on all Append() calls
I20260810 18:47:48.127912 6633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:48.115070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56159 (local address 127.4.228.254:33731)
0810 18:47:48.115977 (+ 907us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:48.115990 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:48.116033 (+ 43us) server_negotiation.cc:407] Connection header received
0810 18:47:48.117863 (+ 1830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:48.117888 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:48.118305 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:48.118734 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:48.119403 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:48.120825 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:48.122789 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:48.123594 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:48.123917 (+ 323us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:48.125294 (+ 1377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:48.125303 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:48.125319 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:47:48.125969 (+ 650us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:48.126085 (+ 116us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:47:48.126116 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:48.126269 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:48.126573 (+ 304us) server_negotiation.cc:299] Negotiation successful
0810 18:47:48.126870 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":678,"thread_start_us":429,"threads_started":1}
I20260810 18:47:48.127961 6630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2144bf89790344648965aaaf21e26928: No bootstrap required, opened a new log
I20260810 18:47:48.132136 5011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:47:48.137647 6630 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2144bf89790344648965aaaf21e26928 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2144bf89790344648965aaaf21e26928" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 33731 } }
I20260810 18:47:48.138274 6630 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2144bf89790344648965aaaf21e26928 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:47:48.138445 6630 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2144bf89790344648965aaaf21e26928 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2144bf89790344648965aaaf21e26928, State: Initialized, Role: FOLLOWER
I20260810 18:47:48.139259 6630 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2144bf89790344648965aaaf21e26928 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2144bf89790344648965aaaf21e26928" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 33731 } }
I20260810 18:47:48.139602 6630 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2144bf89790344648965aaaf21e26928 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:47:48.139765 6630 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2144bf89790344648965aaaf21e26928 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:47:48.139981 6630 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2144bf89790344648965aaaf21e26928 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:47:48.143257 6630 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2144bf89790344648965aaaf21e26928 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2144bf89790344648965aaaf21e26928" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 33731 } }
I20260810 18:47:48.144019 6630 leader_election.cc:302] T 00000000000000000000000000000000 P 2144bf89790344648965aaaf21e26928 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2144bf89790344648965aaaf21e26928; no voters:
I20260810 18:47:48.144697 6633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:48.136984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38403 (local address 127.4.228.254:33731)
0810 18:47:48.137233 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:48.137241 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:48.137774 (+ 533us) server_negotiation.cc:407] Connection header received
0810 18:47:48.137980 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:48.137986 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:48.138086 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:48.138216 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:48.138957 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:48.140195 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:48.141550 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:48.142327 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:48.142494 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:48.143767 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:48.143777 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:48.143783 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:47:48.143807 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:48.143858 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:47:48.143881 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:48.143998 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:48.144154 (+ 156us) server_negotiation.cc:299] Negotiation successful
0810 18:47:48.144261 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260810 18:47:48.144865 6630 leader_election.cc:288] T 00000000000000000000000000000000 P 2144bf89790344648965aaaf21e26928 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:47:48.145071 6641 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2144bf89790344648965aaaf21e26928 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:47:48.147203 6641 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2144bf89790344648965aaaf21e26928 [term 1 LEADER]: Becoming Leader. State: Replica: 2144bf89790344648965aaaf21e26928, State: Running, Role: LEADER
I20260810 18:47:48.147780 6630 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2144bf89790344648965aaaf21e26928 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:47:48.147830 6641 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2144bf89790344648965aaaf21e26928 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2144bf89790344648965aaaf21e26928" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 33731 } }
I20260810 18:47:48.151785 6642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2144bf89790344648965aaaf21e26928 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2144bf89790344648965aaaf21e26928" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2144bf89790344648965aaaf21e26928" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 33731 } } }
I20260810 18:47:48.151815 6643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2144bf89790344648965aaaf21e26928 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2144bf89790344648965aaaf21e26928. Latest consensus state: current_term: 1 leader_uuid: "2144bf89790344648965aaaf21e26928" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2144bf89790344648965aaaf21e26928" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 33731 } } }
I20260810 18:47:48.152293 6642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2144bf89790344648965aaaf21e26928 [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:48.152293 6643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2144bf89790344648965aaaf21e26928 [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:48.156054 6652 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:47:48.161435 6652 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:47:48.171079 6652 catalog_manager.cc:1347] Generated new cluster ID: 411126f9f2eb4c93b3dfcc007cb58f17
I20260810 18:47:48.171221 6652 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:47:48.205044 6652 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:47:48.206645 6652 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:47:48.220827 6652 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2144bf89790344648965aaaf21e26928: Generated new TSK 0
I20260810 18:47:48.221766 6652 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:47:48.235464 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 6530
W20260810 18:47:48.236721 6622 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:56159
W20260810 18:47:48.237558 5011 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 6530: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:47:48.237730 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 6530
2026-08-10T18:47:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2434 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:47:48 dist-test-slave-7kzw krb5kdc[6669](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:47:48 dist-test-slave-7kzw krb5kdc[6669](info): set up 2 sockets
Aug 10 18:47:48 dist-test-slave-7kzw krb5kdc[6669](info): commencing operation
krb5kdc: starting...
W20260810 18:47:49.638311 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.351s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:47:49 dist-test-slave-7kzw krb5kdc[6669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387669, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:47:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:47:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.228.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
I20260810 18:47:50.030005 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:34127
--webserver_interface=127.4.228.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:42585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:34127
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:47:50.437024 6691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:47:50.437402 6691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:47:50.437572 6691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:47:50.445590 6691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:47:50.445756 6691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:47:50.445859 6691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:47:50.445971 6691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:47:50.446066 6691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:47:50.458004 6691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:42585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:34127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:34127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:47:50.459830 6691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:47:50.461423 6691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:47:50.475637 6699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:50.475768 6701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:50.476336 6698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:47:50.477512 6691 server_base.cc:1034] running on GCE node
I20260810 18:47:50.478622 6691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:47:50.480458 6691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:47:50.481832 6691 hybrid_clock.cc:648] HybridClock initialized: now 1786387670481780 us; error 40 us; skew 500 ppm
I20260810 18:47:50.482402 6691 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:47:50 dist-test-slave-7kzw krb5kdc[6669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387670, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:47:50.492444 6691 init.cc:377] Logged in from keytab as kudu/127.4.228.254@KRBTEST.COM (short username kudu)
I20260810 18:47:50.494445 6691 webserver.cc:460] Webserver started at http://127.4.228.254:33157/ using document root <none> and password file <none>
I20260810 18:47:50.495347 6691 fs_manager.cc:359] Metadata directory not provided
I20260810 18:47:50.495514 6691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:47:50.495880 6691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:47:50.499661 6691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "9170dd9434c6444c98ab7c7e981c8eeb"
format_stamp: "Formatted at 2026-08-10 18:47:50 on dist-test-slave-7kzw"
server_key: "51e6b717d4f43b57f82287ea177c0c87"
server_key_iv: "35417bbe29dd83c2a8fbd30e2df82799"
server_key_version: "encryptionkey@0"
I20260810 18:47:50.500784 6691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "9170dd9434c6444c98ab7c7e981c8eeb"
format_stamp: "Formatted at 2026-08-10 18:47:50 on dist-test-slave-7kzw"
server_key: "51e6b717d4f43b57f82287ea177c0c87"
server_key_iv: "35417bbe29dd83c2a8fbd30e2df82799"
server_key_version: "encryptionkey@0"
I20260810 18:47:50.508044 6691 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260810 18:47:50.513367 6712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:50.515213 6691 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260810 18:47:50.515437 6691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "9170dd9434c6444c98ab7c7e981c8eeb"
format_stamp: "Formatted at 2026-08-10 18:47:50 on dist-test-slave-7kzw"
server_key: "51e6b717d4f43b57f82287ea177c0c87"
server_key_iv: "35417bbe29dd83c2a8fbd30e2df82799"
server_key_version: "encryptionkey@0"
I20260810 18:47:50.515674 6691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:47:50.522651 6691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:47:50.528396 6691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:47:50.528774 6691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:47:50.551393 6691 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:34127
I20260810 18:47:50.551414 6790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:34127 every 8 connection(s)
I20260810 18:47:50.553164 6691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:47:50.555449 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 6691
I20260810 18:47:50.555662 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:47:50.556305 5011 external_mini_cluster.cc:1428] Setting key 7bcc9d3dfede117dd208adc03d5626ad
I20260810 18:47:50.558789 6791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:47:50.565672 6799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:50.558406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33433 (local address 127.4.228.254:34127)
0810 18:47:50.559393 (+ 987us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:50.559406 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:50.559453 (+ 47us) server_negotiation.cc:407] Connection header received
0810 18:47:50.561314 (+ 1861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:50.561338 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:50.561735 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:50.562110 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:50.563707 (+ 1597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:50.563714 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:50.563731 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:47:50.563751 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:50.563821 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:47:50.563844 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:50.564033 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:50.564197 (+ 164us) server_negotiation.cc:299] Negotiation successful
0810 18:47:50.564541 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":732,"thread_start_us":460,"threads_started":1}
I20260810 18:47:50.570124 5011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:47:50.572657 6791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9170dd9434c6444c98ab7c7e981c8eeb: Bootstrap starting.
I20260810 18:47:50.576679 6791 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9170dd9434c6444c98ab7c7e981c8eeb: Neither blocks nor log segments found. Creating new log.
I20260810 18:47:50.578503 6791 log.cc:824] T 00000000000000000000000000000000 P 9170dd9434c6444c98ab7c7e981c8eeb: Log is configured to *not* fsync() on all Append() calls
I20260810 18:47:50.579152 6799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:50.575657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40975 (local address 127.4.228.254:34127)
0810 18:47:50.576036 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:50.576042 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:50.576694 (+ 652us) server_negotiation.cc:407] Connection header received
0810 18:47:50.576973 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:50.576978 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:50.577046 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:50.577186 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:50.578274 (+ 1088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:50.578282 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:50.578287 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:47:50.578307 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:50.578357 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:47:50.578377 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:50.578543 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:50.578772 (+ 229us) server_negotiation.cc:299] Negotiation successful
0810 18:47:50.578849 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235}
I20260810 18:47:50.582427 6791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9170dd9434c6444c98ab7c7e981c8eeb: No bootstrap required, opened a new log
I20260810 18:47:50.590850 6791 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9170dd9434c6444c98ab7c7e981c8eeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9170dd9434c6444c98ab7c7e981c8eeb" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 34127 } }
I20260810 18:47:50.591262 6791 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9170dd9434c6444c98ab7c7e981c8eeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:47:50.591374 6791 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9170dd9434c6444c98ab7c7e981c8eeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9170dd9434c6444c98ab7c7e981c8eeb, State: Initialized, Role: FOLLOWER
I20260810 18:47:50.592084 6791 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9170dd9434c6444c98ab7c7e981c8eeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9170dd9434c6444c98ab7c7e981c8eeb" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 34127 } }
I20260810 18:47:50.592362 6791 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9170dd9434c6444c98ab7c7e981c8eeb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:47:50.592562 6791 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9170dd9434c6444c98ab7c7e981c8eeb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:47:50.592770 6791 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9170dd9434c6444c98ab7c7e981c8eeb [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:47:50.595224 6791 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9170dd9434c6444c98ab7c7e981c8eeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9170dd9434c6444c98ab7c7e981c8eeb" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 34127 } }
I20260810 18:47:50.595826 6791 leader_election.cc:302] T 00000000000000000000000000000000 P 9170dd9434c6444c98ab7c7e981c8eeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9170dd9434c6444c98ab7c7e981c8eeb; no voters:
I20260810 18:47:50.596552 6791 leader_election.cc:288] T 00000000000000000000000000000000 P 9170dd9434c6444c98ab7c7e981c8eeb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:47:50.596750 6810 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9170dd9434c6444c98ab7c7e981c8eeb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:47:50.597465 6810 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9170dd9434c6444c98ab7c7e981c8eeb [term 1 LEADER]: Becoming Leader. State: Replica: 9170dd9434c6444c98ab7c7e981c8eeb, State: Running, Role: LEADER
I20260810 18:47:50.598110 6810 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9170dd9434c6444c98ab7c7e981c8eeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9170dd9434c6444c98ab7c7e981c8eeb" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 34127 } }
I20260810 18:47:50.598950 6791 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9170dd9434c6444c98ab7c7e981c8eeb [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:47:50.601739 6811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9170dd9434c6444c98ab7c7e981c8eeb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9170dd9434c6444c98ab7c7e981c8eeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9170dd9434c6444c98ab7c7e981c8eeb" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 34127 } } }
I20260810 18:47:50.601963 6812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9170dd9434c6444c98ab7c7e981c8eeb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9170dd9434c6444c98ab7c7e981c8eeb. Latest consensus state: current_term: 1 leader_uuid: "9170dd9434c6444c98ab7c7e981c8eeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9170dd9434c6444c98ab7c7e981c8eeb" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 34127 } } }
I20260810 18:47:50.602196 6811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9170dd9434c6444c98ab7c7e981c8eeb [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:50.602252 6812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9170dd9434c6444c98ab7c7e981c8eeb [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:50.604187 6823 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:47:50.610576 6823 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:47:50.621556 6823 catalog_manager.cc:1347] Generated new cluster ID: 7a9b1c542dcc45e38bbb2d888edfd003
I20260810 18:47:50.621712 6823 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:47:50.633975 6823 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:47:50.634434 6823 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:47:50.642729 6823 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9170dd9434c6444c98ab7c7e981c8eeb: Generated new TSK 0
I20260810 18:47:50.643936 6823 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:47:50.656359 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 6691
W20260810 18:47:50.657537 6785 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:33433
W20260810 18:47:50.658280 5011 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 6691: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 18:47:50.658417 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 6691
2026-08-10T18:47:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2427 ms)
[----------] 18 tests from AuthTokenIssuingTest (43173 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:47:50 dist-test-slave-7kzw krb5kdc[6840](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:47:50 dist-test-slave-7kzw krb5kdc[6840](info): set up 2 sockets
Aug 10 18:47:50 dist-test-slave-7kzw krb5kdc[6840](info): commencing operation
krb5kdc: starting...
W20260810 18:47:52.101488 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.385s user 0.006s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:47:52 dist-test-slave-7kzw krb5kdc[6840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:47:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:47:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.228.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
I20260810 18:47:52.526561 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:42879
--webserver_interface=127.4.228.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:45503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:42879,127.4.228.253:45813,127.4.228.252:39279
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:47:52.943822 6861 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 18:47:52.944252 6861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:47:52.944372 6861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:47:52.944486 6861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:47:52.952943 6861 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:47:52.953068 6861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:47:52.953173 6861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:47:52.953269 6861 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:47:52.953373 6861 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:47:52.963958 6861 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:45503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:42879,127.4.228.253:45813,127.4.228.252:39279
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:42879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:47:52.965469 6861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:47:52.966974 6861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:47:52.980931 6872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:52.980960 6869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:52.980981 6867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:47:52.982273 6861 server_base.cc:1034] running on GCE node
I20260810 18:47:52.983361 6861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:47:52.985121 6861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:47:52.986534 6861 hybrid_clock.cc:648] HybridClock initialized: now 1786387672986391 us; error 36 us; skew 500 ppm
I20260810 18:47:52.986995 6861 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:47:52 dist-test-slave-7kzw krb5kdc[6840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387672, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:47:52.990840 6861 init.cc:377] Logged in from keytab as kudu/127.4.228.254@KRBTEST.COM (short username kudu)
I20260810 18:47:52.992604 6861 webserver.cc:460] Webserver started at http://127.4.228.254:37983/ using document root <none> and password file <none>
I20260810 18:47:52.993422 6861 fs_manager.cc:359] Metadata directory not provided
I20260810 18:47:52.993572 6861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:47:52.993933 6861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:47:52.997363 6861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "f565a2cbf6134d42969b79bce9f76b79"
format_stamp: "Formatted at 2026-08-10 18:47:52 on dist-test-slave-7kzw"
server_key: "7a2e1a1c4998113a2649103552754b8d"
server_key_iv: "49018caa1fe579d027554007b4bd276e"
server_key_version: "encryptionkey@0"
I20260810 18:47:52.998251 6861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "f565a2cbf6134d42969b79bce9f76b79"
format_stamp: "Formatted at 2026-08-10 18:47:52 on dist-test-slave-7kzw"
server_key: "7a2e1a1c4998113a2649103552754b8d"
server_key_iv: "49018caa1fe579d027554007b4bd276e"
server_key_version: "encryptionkey@0"
I20260810 18:47:53.005055 6861 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 18:47:53.009538 6884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:53.011355 6861 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260810 18:47:53.011591 6861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "f565a2cbf6134d42969b79bce9f76b79"
format_stamp: "Formatted at 2026-08-10 18:47:52 on dist-test-slave-7kzw"
server_key: "7a2e1a1c4998113a2649103552754b8d"
server_key_iv: "49018caa1fe579d027554007b4bd276e"
server_key_version: "encryptionkey@0"
I20260810 18:47:53.011868 6861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:47:53.039419 6861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:47:53.044370 6861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:47:53.044719 6861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:47:53.066237 6861 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:42879
I20260810 18:47:53.066262 6965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:42879 every 8 connection(s)
I20260810 18:47:53.067806 6861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:47:53.069870 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 6861
I20260810 18:47:53.070132 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:47:53.070894 5011 external_mini_cluster.cc:1428] Setting key 5004303663b23b100c633a1f785f61a7
I20260810 18:47:53.072285 6966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:47:53.079545 6966 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42879 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:47:53 dist-test-slave-7kzw krb5kdc[6840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387672, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:47:53.108350 6971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:53.083044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53082 (local address 127.4.228.254:42879)
0810 18:47:53.084778 (+ 1734us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:53.084787 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:53.084970 (+ 183us) server_negotiation.cc:407] Connection header received
0810 18:47:53.087008 (+ 2038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:53.087021 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:53.087284 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:53.087509 (+ 225us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:53.088918 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:53.090314 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:53.091887 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:53.093130 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:53.093378 (+ 248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:53.097400 (+ 4022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:53.097417 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:53.097440 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:47:53.097493 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:53.102458 (+ 4965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:53.103418 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:53.103433 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:53.103439 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:53.103793 (+ 354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:53.104660 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:53.104664 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:53.104667 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:53.105431 (+ 764us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:53.105689 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:53.106545 (+ 856us) server_negotiation.cc:299] Negotiation successful
0810 18:47:53.107092 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":254,"server-negotiator.queue_time_us":1508,"spinlock_wait_cycles":33152,"thread_start_us":1278,"threads_started":1}
I20260810 18:47:53.108418 6969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:53.083043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:42879 (local address 127.0.0.1:53082)
0810 18:47:53.083961 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:53.084019 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 18:47:53.086421 (+ 2402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:47:53.087535 (+ 1114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:47:53.087542 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:47:53.088326 (+ 784us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:47:53.088679 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:53.088698 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:53.090519 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:53.090529 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:53.091628 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:53.091642 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:53.093404 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:53.093410 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:53.093544 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:53.094391 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:47:53.094419 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:47:53.097092 (+ 2673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:47:53.102700 (+ 5608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:47:53.102713 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:47:53.102726 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:47:53.103227 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:47:53.104196 (+ 969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:47:53.104205 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:47:53.104212 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:47:53.104479 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:47:53.105629 (+ 1150us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:47:53.105642 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:47:53.105936 (+ 294us) client_negotiation.cc:769] Sending connection context
0810 18:47:53.106273 (+ 337us) client_negotiation.cc:240] Negotiation successful
0810 18:47:53.106931 (+ 658us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":389,"threads_started":1}
WARNING: no policy specified for kudu/127.4.228.253@KRBTEST.COM; defaulting to no policy
I20260810 18:47:53.111737 6966 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.228.253" port: 45813 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.253@KRBTEST.COM" created.
I20260810 18:47:53.113209 6969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:53.112619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:45813 (local address 127.0.0.1:36264)
0810 18:47:53.112858 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:53.113028 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.253:45813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260810 18:47:53.113550 6887 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.228.253:45813: connect: Connection refused (error 111)
I20260810 18:47:53.115494 6969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:53.114918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:45813 (local address 127.0.0.1:36276)
0810 18:47:53.115139 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:53.115262 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.253:45813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260810 18:47:53.115768 6966 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.253:45813: Network error: Client connection negotiation failed: client connection to 127.4.228.253:45813: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.253.keytab.
Entry for principal kudu/127.4.228.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.253.keytab.
I20260810 18:47:53.159117 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.253:45813
--webserver_interface=127.4.228.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.253.keytab
--principal=kudu/127.4.228.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:45503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:42879,127.4.228.253:45813,127.4.228.252:39279
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:47:53.165218 6966 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.253" port: 45813 } attempt: 1
I20260810 18:47:53.166785 6969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:53.166077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:45813 (local address 127.0.0.1:36278)
0810 18:47:53.166345 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:53.166554 (+ 209us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.253:45813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260810 18:47:53.168201 6969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:53.167650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:45813 (local address 127.0.0.1:36282)
0810 18:47:53.167902 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:53.168008 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.253:45813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260810 18:47:53.168469 6966 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.253:45813: Network error: Client connection negotiation failed: client connection to 127.4.228.253:45813: connect: Connection refused (error 111)
I20260810 18:47:53.236850 6966 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.253" port: 45813 } attempt: 2
I20260810 18:47:53.238317 6969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:53.237705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:45813 (local address 127.0.0.1:36290)
0810 18:47:53.238035 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:53.238154 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.253:45813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260810 18:47:53.239661 6969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:53.239166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:45813 (local address 127.0.0.1:36294)
0810 18:47:53.239398 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:53.239503 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.253:45813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260810 18:47:53.239964 6966 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.253:45813: Network error: Client connection negotiation failed: client connection to 127.4.228.253:45813: connect: Connection refused (error 111)
I20260810 18:47:53.331274 6966 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.253" port: 45813 } attempt: 3
I20260810 18:47:53.332720 6969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:53.332181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:45813 (local address 127.0.0.1:48132)
0810 18:47:53.332435 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:53.332536 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.253:45813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260810 18:47:53.334052 6969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:53.333612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:45813 (local address 127.0.0.1:48144)
0810 18:47:53.333794 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:53.333886 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.253:45813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260810 18:47:53.334292 6966 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.253:45813: Network error: Client connection negotiation failed: client connection to 127.4.228.253:45813: connect: Connection refused (error 111)
I20260810 18:47:53.477735 6966 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.253" port: 45813 } attempt: 4
I20260810 18:47:53.479513 6969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:53.478753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:45813 (local address 127.0.0.1:48158)
0810 18:47:53.479251 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:53.479355 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.253:45813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224}
I20260810 18:47:53.481803 6969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:53.481103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:45813 (local address 127.0.0.1:48164)
0810 18:47:53.481334 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:53.481446 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.253:45813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260810 18:47:53.482295 6966 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.253:45813: Network error: Client connection negotiation failed: client connection to 127.4.228.253:45813: connect: Connection refused (error 111)
W20260810 18:47:53.557490 6976 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 18:47:53.557926 6976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:47:53.557999 6976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:47:53.558178 6976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:47:53.568487 6976 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:47:53.568624 6976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:47:53.568698 6976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:47:53.568858 6976 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:47:53.568967 6976 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:47:53.581869 6976 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:45503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:42879,127.4.228.253:45813,127.4.228.252:39279
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.253.keytab
--principal=kudu/127.4.228.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.253:45813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.228.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:47:53.583596 6976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:47:53.585270 6976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:47:53.599435 6990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:53.599536 6985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:53.599521 6987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:47:53.601948 6976 server_base.cc:1034] running on GCE node
I20260810 18:47:53.603152 6976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:47:53.605140 6976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:47:53.608695 6976 hybrid_clock.cc:648] HybridClock initialized: now 1786387673608485 us; error 87 us; skew 500 ppm
I20260810 18:47:53.609380 6976 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:47:53 dist-test-slave-7kzw krb5kdc[6840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387673, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:47:53.619240 6976 init.cc:377] Logged in from keytab as kudu/127.4.228.253@KRBTEST.COM (short username kudu)
I20260810 18:47:53.621172 6976 webserver.cc:460] Webserver started at http://127.4.228.253:34061/ using document root <none> and password file <none>
I20260810 18:47:53.622316 6976 fs_manager.cc:359] Metadata directory not provided
I20260810 18:47:53.622529 6976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:47:53.622948 6976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:47:53.626852 6976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-1/data/instance:
uuid: "f9b2366575374f49aee0f2ebf612670b"
format_stamp: "Formatted at 2026-08-10 18:47:53 on dist-test-slave-7kzw"
server_key: "3eea133641d0372f52c3049c314c1c9e"
server_key_iv: "c9d82bfd58a9430aad4cf86639535ea0"
server_key_version: "encryptionkey@0"
I20260810 18:47:53.628243 6976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-1/wal/instance:
uuid: "f9b2366575374f49aee0f2ebf612670b"
format_stamp: "Formatted at 2026-08-10 18:47:53 on dist-test-slave-7kzw"
server_key: "3eea133641d0372f52c3049c314c1c9e"
server_key_iv: "c9d82bfd58a9430aad4cf86639535ea0"
server_key_version: "encryptionkey@0"
I20260810 18:47:53.634460 6976 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260810 18:47:53.639416 6999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:53.641450 6976 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260810 18:47:53.641809 6976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-1/wal
uuid: "f9b2366575374f49aee0f2ebf612670b"
format_stamp: "Formatted at 2026-08-10 18:47:53 on dist-test-slave-7kzw"
server_key: "3eea133641d0372f52c3049c314c1c9e"
server_key_iv: "c9d82bfd58a9430aad4cf86639535ea0"
server_key_version: "encryptionkey@0"
I20260810 18:47:53.642134 6976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:47:53.674685 6976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:47:53.679720 6976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:47:53.680037 6976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:47:53.707860 6976 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.253:45813
I20260810 18:47:53.707886 7083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.253:45813 every 8 connection(s)
I20260810 18:47:53.709653 6976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-1/data/info.pb
I20260810 18:47:53.711956 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 6976
I20260810 18:47:53.712203 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-1/wal/instance
I20260810 18:47:53.712849 5011 external_mini_cluster.cc:1428] Setting key 14c0391c6bfa1d0578e92eb61b6636b4
I20260810 18:47:53.715683 7084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:47:53.727571 7084 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42879 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:47:53 dist-test-slave-7kzw krb5kdc[6840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387673, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.253@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.228.252@KRBTEST.COM; defaulting to no policy
I20260810 18:47:53.753592 7087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:53.731538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51388 (local address 127.4.228.254:42879)
0810 18:47:53.732110 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:53.732115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:53.733474 (+ 1359us) server_negotiation.cc:407] Connection header received
0810 18:47:53.735809 (+ 2335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:53.735815 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:53.735921 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:53.736067 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:53.738002 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:53.738894 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:53.740829 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:53.741260 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:53.741408 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:53.745781 (+ 4373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:53.745787 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:53.745791 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:47:53.745863 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:53.749456 (+ 3593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:53.750649 (+ 1193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:53.750654 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:53.750657 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:53.750871 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:53.751402 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:53.751408 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:53.751412 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:53.751778 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:53.751966 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:53.752860 (+ 894us) server_negotiation.cc:299] Negotiation successful
0810 18:47:53.753136 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":317,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.252@KRBTEST.COM" created.
I20260810 18:47:53.754969 7088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:53.732322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:42879 (local address 127.0.0.1:51388)
0810 18:47:53.733257 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:53.733315 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 18:47:53.735457 (+ 2142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:47:53.736375 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:47:53.736386 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:47:53.737020 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:47:53.737739 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:53.737761 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:53.739094 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:53.739098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:53.740630 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:53.740645 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:53.741521 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:53.741530 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:53.741791 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:53.742604 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:47:53.742630 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:47:53.745497 (+ 2867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:47:53.749741 (+ 4244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:47:53.749758 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:47:53.749776 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:47:53.750395 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:47:53.751008 (+ 613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:47:53.751015 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:47:53.751019 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:47:53.751253 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:47:53.751936 (+ 683us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:47:53.751946 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:47:53.752413 (+ 467us) client_negotiation.cc:769] Sending connection context
0810 18:47:53.752726 (+ 313us) client_negotiation.cc:240] Negotiation successful
0810 18:47:53.753371 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":410,"threads_started":1}
I20260810 18:47:53.757191 7084 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.228.253" port: 45813 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:47:53 dist-test-slave-7kzw krb5kdc[6840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387673, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.253@KRBTEST.COM for kudu/127.4.228.253@KRBTEST.COM
I20260810 18:47:53.781605 6966 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.253" port: 45813 } attempt: 5
I20260810 18:47:53.787868 7088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:53.758345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:45813 (local address 127.0.0.1:48172)
0810 18:47:53.758619 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:53.758644 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 18:47:53.758783 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:47:53.760231 (+ 1448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:47:53.760240 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:47:53.761220 (+ 980us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:47:53.761477 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:53.761494 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:53.763296 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:53.763305 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:53.766741 (+ 3436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:53.768088 (+ 1347us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:53.769867 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:53.769878 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:53.770105 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:53.771245 (+ 1140us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:47:53.771276 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:47:53.775485 (+ 4209us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:47:53.781160 (+ 5675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:47:53.781165 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:47:53.781169 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:47:53.783244 (+ 2075us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:47:53.784179 (+ 935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:47:53.784188 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:47:53.784195 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:47:53.784453 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:47:53.786248 (+ 1795us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:47:53.786257 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:47:53.786582 (+ 325us) client_negotiation.cc:769] Sending connection context
0810 18:47:53.786771 (+ 189us) client_negotiation.cc:240] Negotiation successful
0810 18:47:53.787157 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78,"mutex_wait_us":29}
I20260810 18:47:53.789469 7089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:53.758488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48172 (local address 127.4.228.253:45813)
0810 18:47:53.759569 (+ 1081us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:53.759577 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:53.759605 (+ 28us) server_negotiation.cc:407] Connection header received
0810 18:47:53.759655 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:53.759661 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:53.759868 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:53.760009 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:53.761755 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:53.763124 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:53.768384 (+ 5260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:53.769572 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:53.769731 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:53.775901 (+ 6170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:53.775928 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:53.775955 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:47:53.776006 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:53.780898 (+ 4892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:53.783466 (+ 2568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:53.783475 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:53.783478 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:53.783989 (+ 511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:53.784627 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:53.784632 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:53.784635 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:53.786007 (+ 1372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:53.786431 (+ 424us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:53.787788 (+ 1357us) server_negotiation.cc:299] Negotiation successful
0810 18:47:53.788912 (+ 1124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":157,"server-negotiator.queue_time_us":879,"thread_start_us":723,"threads_started":1}
I20260810 18:47:53.793148 7084 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.228.252" port: 39279 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:47:53.795071 7088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:53.793976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:39279 (local address 127.0.0.1:37376)
0810 18:47:53.794312 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:53.794454 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:39279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165}
W20260810 18:47:53.795588 7001 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.228.252:39279: connect: Connection refused (error 111)
Aug 10 18:47:53 dist-test-slave-7kzw krb5kdc[6840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387672, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for kudu/127.4.228.253@KRBTEST.COM
I20260810 18:47:53.800463 7088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:53.799873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:39279 (local address 127.0.0.1:37380)
0810 18:47:53.800097 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:53.800252 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:39279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260810 18:47:53.801095 7084 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.252:39279: Network error: Client connection negotiation failed: client connection to 127.4.228.252:39279: connect: Connection refused (error 111)
I20260810 18:47:53.810359 6969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:53.782372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:45813 (local address 127.0.0.1:48180)
0810 18:47:53.783118 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:53.783148 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 18:47:53.783290 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:47:53.784872 (+ 1582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:47:53.784880 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:47:53.785435 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:47:53.785759 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:53.785778 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:53.789703 (+ 3925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:53.789716 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:53.792072 (+ 2356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:53.792106 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:53.794080 (+ 1974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:53.794088 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:53.794348 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:53.795102 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:47:53.795133 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:47:53.799129 (+ 3996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:47:53.805540 (+ 6411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:47:53.805547 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:47:53.805552 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:47:53.806840 (+ 1288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:47:53.807949 (+ 1109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:47:53.807956 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:47:53.807962 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:47:53.808141 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:47:53.809233 (+ 1092us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:47:53.809238 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:47:53.809431 (+ 193us) client_negotiation.cc:769] Sending connection context
0810 18:47:53.809625 (+ 194us) client_negotiation.cc:240] Negotiation successful
0810 18:47:53.809972 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260810 18:47:53.810439 7092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:53.783225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48180 (local address 127.4.228.253:45813)
0810 18:47:53.784283 (+ 1058us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:53.784287 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:53.784379 (+ 92us) server_negotiation.cc:407] Connection header received
0810 18:47:53.784507 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:53.784512 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:53.784633 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:53.784808 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:53.786007 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:53.789358 (+ 3351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:53.792780 (+ 3422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:53.793846 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:53.794024 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:53.799472 (+ 5448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:53.799480 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:53.799487 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:47:53.799540 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:53.805324 (+ 5784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:53.807033 (+ 1709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:53.807044 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:53.807225 (+ 181us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:53.807469 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:53.808290 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:53.808299 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:53.808307 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:53.809060 (+ 753us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:53.809281 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:53.809763 (+ 482us) server_negotiation.cc:299] Negotiation successful
0810 18:47:53.810003 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":218,"server-negotiator.queue_time_us":909,"thread_start_us":672,"threads_started":1}
I20260810 18:47:53.812438 6966 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.228.252" port: 39279 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:47:53.813760 6969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:53.813251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:39279 (local address 127.0.0.1:37394)
0810 18:47:53.813476 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:53.813578 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:39279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260810 18:47:53.815604 6969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:53.814829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:39279 (local address 127.0.0.1:37400)
0810 18:47:53.815314 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:53.815453 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:39279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354}
W20260810 18:47:53.816103 6966 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.252:39279: Network error: Client connection negotiation failed: client connection to 127.4.228.252:39279: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.252.keytab.
Entry for principal kudu/127.4.228.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.252.keytab.
W20260810 18:47:53.826817 5011 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.4.228.252: real 0.113s user 0.001s sys 0.040s
I20260810 18:47:53.827778 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.252:39279
--webserver_interface=127.4.228.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.252.keytab
--principal=kudu/127.4.228.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:45503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:42879,127.4.228.253:45813,127.4.228.252:39279
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:47:53.850554 7084 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.252" port: 39279 } attempt: 1
I20260810 18:47:53.852245 7088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:53.851588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:39279 (local address 127.0.0.1:37408)
0810 18:47:53.851868 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:53.852013 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:39279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260810 18:47:53.854224 7088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:53.853298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:39279 (local address 127.0.0.1:37414)
0810 18:47:53.853529 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:53.853854 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:39279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260810 18:47:53.854708 7084 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.252:39279: Network error: Client connection negotiation failed: client connection to 127.4.228.252:39279: connect: Connection refused (error 111)
I20260810 18:47:53.867614 6966 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.252" port: 39279 } attempt: 1
I20260810 18:47:53.869385 6969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:53.868692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:39279 (local address 127.0.0.1:37420)
0810 18:47:53.868941 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:53.869080 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:39279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260810 18:47:53.870651 6969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:53.870178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:39279 (local address 127.0.0.1:37424)
0810 18:47:53.870355 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:53.870451 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:39279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260810 18:47:53.870976 6966 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.252:39279: Network error: Client connection negotiation failed: client connection to 127.4.228.252:39279: connect: Connection refused (error 111)
I20260810 18:47:53.923005 7084 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.252" port: 39279 } attempt: 2
I20260810 18:47:53.924572 7088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:53.923975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:39279 (local address 127.0.0.1:37430)
0810 18:47:53.924292 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:53.924393 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:39279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172}
I20260810 18:47:53.926033 7088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:53.925498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:39279 (local address 127.0.0.1:37446)
0810 18:47:53.925773 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:53.925876 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:39279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260810 18:47:53.926306 7084 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.252:39279: Network error: Client connection negotiation failed: client connection to 127.4.228.252:39279: connect: Connection refused (error 111)
I20260810 18:47:53.939293 6966 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.252" port: 39279 } attempt: 2
I20260810 18:47:53.940634 6969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:53.940088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:39279 (local address 127.0.0.1:37450)
0810 18:47:53.940357 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:53.940450 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:39279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260810 18:47:53.942003 6969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:53.941344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:39279 (local address 127.0.0.1:37456)
0810 18:47:53.941516 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:53.941874 (+ 358us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:39279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 18:47:53.942386 6966 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.252:39279: Network error: Client connection negotiation failed: client connection to 127.4.228.252:39279: connect: Connection refused (error 111)
I20260810 18:47:54.017555 7084 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.252" port: 39279 } attempt: 3
I20260810 18:47:54.019035 7088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:54.018318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:39279 (local address 127.0.0.1:37468)
0810 18:47:54.018568 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:54.018668 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:39279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260810 18:47:54.020408 7088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:54.019865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:39279 (local address 127.0.0.1:37470)
0810 18:47:54.020141 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:54.020223 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:39279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
W20260810 18:47:54.020648 7084 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.252:39279: Network error: Client connection negotiation failed: client connection to 127.4.228.252:39279: connect: Connection refused (error 111)
I20260810 18:47:54.048674 6966 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.252" port: 39279 } attempt: 3
I20260810 18:47:54.050268 6969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:54.049469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:39279 (local address 127.0.0.1:37480)
0810 18:47:54.049758 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:54.049857 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:39279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260810 18:47:54.051654 6969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:54.051162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:39279 (local address 127.0.0.1:37496)
0810 18:47:54.051375 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:54.051458 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:39279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260810 18:47:54.051930 6966 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.252:39279: Network error: Client connection negotiation failed: client connection to 127.4.228.252:39279: connect: Connection refused (error 111)
I20260810 18:47:54.164057 7084 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.252" port: 39279 } attempt: 4
I20260810 18:47:54.165680 7088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:54.165074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:39279 (local address 127.0.0.1:37500)
0810 18:47:54.165374 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:54.165494 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:39279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
I20260810 18:47:54.166978 7088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:54.166595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:39279 (local address 127.0.0.1:37512)
0810 18:47:54.166759 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:54.166853 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:39279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 18:47:54.167212 7084 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.252:39279: Network error: Client connection negotiation failed: client connection to 127.4.228.252:39279: connect: Connection refused (error 111)
I20260810 18:47:54.229228 6966 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.252" port: 39279 } attempt: 4
I20260810 18:47:54.230800 6969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:54.230261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:39279 (local address 127.0.0.1:37528)
0810 18:47:54.230524 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:54.230622 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:39279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260810 18:47:54.231918 6969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:54.231521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:39279 (local address 127.0.0.1:37540)
0810 18:47:54.231716 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:54.231790 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:39279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260810 18:47:54.232146 6966 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.252:39279: Network error: Client connection negotiation failed: client connection to 127.4.228.252:39279: connect: Connection refused (error 111)
W20260810 18:47:54.369117 7096 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 18:47:54.369575 7096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:47:54.369714 7096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:47:54.369858 7096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:47:54.377786 7096 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:47:54.377954 7096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:47:54.378088 7096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:47:54.378176 7096 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:47:54.378293 7096 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:47:54.390031 7096 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:45503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:42879,127.4.228.253:45813,127.4.228.252:39279
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.252.keytab
--principal=kudu/127.4.228.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.252:39279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.228.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:47:54.391595 7096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:47:54.393146 7096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:47:54.408644 7102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:54.408888 7101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:54.408915 7105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:47:54.409842 7096 server_base.cc:1034] running on GCE node
I20260810 18:47:54.410940 7096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:47:54.412745 7096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:47:54.414039 7096 hybrid_clock.cc:648] HybridClock initialized: now 1786387674413971 us; error 48 us; skew 500 ppm
I20260810 18:47:54.414426 7096 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:47:54 dist-test-slave-7kzw krb5kdc[6840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387674, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:47:54.418774 7096 init.cc:377] Logged in from keytab as kudu/127.4.228.252@KRBTEST.COM (short username kudu)
I20260810 18:47:54.421022 7096 webserver.cc:460] Webserver started at http://127.4.228.252:40299/ using document root <none> and password file <none>
I20260810 18:47:54.421970 7096 fs_manager.cc:359] Metadata directory not provided
I20260810 18:47:54.422125 7096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:47:54.422587 7096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:47:54.426409 7096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-2/data/instance:
uuid: "078c67ac34374f6fa3b8f7588dc421a1"
format_stamp: "Formatted at 2026-08-10 18:47:54 on dist-test-slave-7kzw"
server_key: "41b9c39be26def42704448a05852bc5b"
server_key_iv: "5e53d2b516836a598b5b3a9e2e451f00"
server_key_version: "encryptionkey@0"
I20260810 18:47:54.427498 7096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-2/wal/instance:
uuid: "078c67ac34374f6fa3b8f7588dc421a1"
format_stamp: "Formatted at 2026-08-10 18:47:54 on dist-test-slave-7kzw"
server_key: "41b9c39be26def42704448a05852bc5b"
server_key_iv: "5e53d2b516836a598b5b3a9e2e451f00"
server_key_version: "encryptionkey@0"
I20260810 18:47:54.434121 7096 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260810 18:47:54.438184 7117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:54.439895 7096 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:47:54.440121 7096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-2/wal
uuid: "078c67ac34374f6fa3b8f7588dc421a1"
format_stamp: "Formatted at 2026-08-10 18:47:54 on dist-test-slave-7kzw"
server_key: "41b9c39be26def42704448a05852bc5b"
server_key_iv: "5e53d2b516836a598b5b3a9e2e451f00"
server_key_version: "encryptionkey@0"
I20260810 18:47:54.440382 7096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:47:54.466559 7084 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.252" port: 39279 } attempt: 5
I20260810 18:47:54.467990 7088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:54.467440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:39279 (local address 127.0.0.1:37542)
0810 18:47:54.467690 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:54.467801 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:39279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260810 18:47:54.469446 7088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:54.468890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:39279 (local address 127.0.0.1:37558)
0810 18:47:54.469087 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:54.469188 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:39279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260810 18:47:54.469815 7084 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.252:39279: Network error: Client connection negotiation failed: client connection to 127.4.228.252:39279: connect: Connection refused (error 111)
I20260810 18:47:54.494823 7096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:47:54.503170 7096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:47:54.503548 7096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:47:54.509436 6966 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.252" port: 39279 } attempt: 5
I20260810 18:47:54.510991 6969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:54.510248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:39279 (local address 127.0.0.1:37574)
0810 18:47:54.510627 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:54.510762 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:39279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176}
I20260810 18:47:54.512353 6969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:54.511884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:39279 (local address 127.0.0.1:37586)
0810 18:47:54.512112 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:54.512223 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:39279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260810 18:47:54.512725 6966 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.252:39279: Network error: Client connection negotiation failed: client connection to 127.4.228.252:39279: connect: Connection refused (error 111)
I20260810 18:47:54.527690 7096 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.252:39279
I20260810 18:47:54.527791 7185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.252:39279 every 8 connection(s)
I20260810 18:47:54.529179 7096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-2/data/info.pb
I20260810 18:47:54.533099 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 7096
I20260810 18:47:54.533396 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387267874959-5011-0/minicluster-data/master-2/wal/instance
I20260810 18:47:54.533895 5011 external_mini_cluster.cc:1428] Setting key 6b93e9b1c847c5685a6e628a72789671
I20260810 18:47:54.534575 7186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:47:54.545487 7186 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42879 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:47:54 dist-test-slave-7kzw krb5kdc[6840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.252@KRBTEST.COM
Aug 10 18:47:54 dist-test-slave-7kzw krb5kdc[6840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387674, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.252@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:47:54.568717 7189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:54.536031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37600 (local address 127.4.228.252:39279)
0810 18:47:54.537131 (+ 1100us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:54.537147 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:54.537199 (+ 52us) server_negotiation.cc:407] Connection header received
0810 18:47:54.540545 (+ 3346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:54.540585 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:54.541103 (+ 518us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:54.541724 (+ 621us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:54.544547 (+ 2823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:54.546694 (+ 2147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:54.549069 (+ 2375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:54.550404 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:54.550635 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:54.556726 (+ 6091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:54.556738 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:54.556759 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:47:54.556822 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:54.560692 (+ 3870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:54.562119 (+ 1427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:54.562127 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:54.562135 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:54.562320 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:54.563297 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:54.563305 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:54.563309 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:54.564172 (+ 863us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:54.564781 (+ 609us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:54.565372 (+ 591us) server_negotiation.cc:299] Negotiation successful
0810 18:47:54.566699 (+ 1327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":862,"thread_start_us":590,"threads_started":1}
I20260810 18:47:54.569373 7190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:54.548981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:42879 (local address 127.0.0.1:51404)
0810 18:47:54.550164 (+ 1183us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:54.550225 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0810 18:47:54.550434 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:47:54.551154 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:47:54.551160 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:47:54.552147 (+ 987us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:47:54.552546 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:54.552563 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:54.553857 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:54.553863 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:54.555319 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:54.555343 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:54.556636 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:54.556646 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:54.556864 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:54.557756 (+ 892us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:47:54.557788 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:47:54.561314 (+ 3526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:47:54.564897 (+ 3583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:47:54.564909 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:47:54.564917 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:47:54.565396 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:47:54.566163 (+ 767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:47:54.566173 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:47:54.566178 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:47:54.566741 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:47:54.567397 (+ 656us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:47:54.567407 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:47:54.567617 (+ 210us) client_negotiation.cc:769] Sending connection context
0810 18:47:54.568125 (+ 508us) client_negotiation.cc:240] Negotiation successful
0810 18:47:54.568551 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":784,"thread_start_us":526,"threads_started":1}
I20260810 18:47:54.570019 7193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:54.549283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51404 (local address 127.4.228.254:42879)
0810 18:47:54.550761 (+ 1478us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:54.550768 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:54.550788 (+ 20us) server_negotiation.cc:407] Connection header received
0810 18:47:54.550853 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:54.550858 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:54.550933 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:54.551024 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:54.552775 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:54.553625 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:54.555529 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:54.556309 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:54.556542 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:54.561721 (+ 5179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:54.561766 (+ 45us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:54.561774 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:47:54.561828 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:54.564739 (+ 2911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:54.565541 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:54.565545 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:54.565547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:54.565747 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:54.566908 (+ 1161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:54.566912 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:54.566915 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:54.567221 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:54.567382 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:54.568484 (+ 1102us) server_negotiation.cc:299] Negotiation successful
0810 18:47:54.568799 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1340,"thread_start_us":1129,"threads_started":1}
I20260810 18:47:54.572983 7186 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.228.253" port: 45813 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:47:54 dist-test-slave-7kzw krb5kdc[6840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387674, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.252@KRBTEST.COM for kudu/127.4.228.253@KRBTEST.COM
Aug 10 18:47:54 dist-test-slave-7kzw krb5kdc[6840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:47:54.592090 7190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:54.574036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:45813 (local address 127.0.0.1:48186)
0810 18:47:54.574501 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:54.574526 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 18:47:54.574679 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:47:54.575332 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:47:54.575337 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:47:54.575933 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:47:54.576160 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:54.576174 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:54.577718 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:54.577725 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:54.578855 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:54.578870 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:54.579716 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:54.579729 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:54.579972 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:54.580672 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:47:54.580698 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:47:54.582910 (+ 2212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:47:54.588082 (+ 5172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:47:54.588089 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:47:54.588093 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:47:54.588698 (+ 605us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:47:54.589361 (+ 663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:47:54.589366 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:47:54.589370 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:47:54.589613 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:47:54.591000 (+ 1387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:47:54.591004 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:47:54.591184 (+ 180us) client_negotiation.cc:769] Sending connection context
0810 18:47:54.591411 (+ 227us) client_negotiation.cc:240] Negotiation successful
0810 18:47:54.591772 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312}
I20260810 18:47:54.592330 7196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:54.574380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48186 (local address 127.4.228.253:45813)
0810 18:47:54.574986 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:54.574992 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:54.575014 (+ 22us) server_negotiation.cc:407] Connection header received
0810 18:47:54.575083 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:54.575087 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:54.575150 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:54.575246 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:54.576362 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:54.577461 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:54.579029 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:54.579468 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:54.579571 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:54.583158 (+ 3587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:54.583168 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:54.583174 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:47:54.583221 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:54.587890 (+ 4669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:54.589048 (+ 1158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:54.589055 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:54.589057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:54.589222 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:54.589964 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:54.589969 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:54.589971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:54.590836 (+ 865us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:54.591020 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:54.591600 (+ 580us) server_negotiation.cc:299] Negotiation successful
0810 18:47:54.591789 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":310,"threads_started":1}
I20260810 18:47:54.594406 7193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:54.574088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51416 (local address 127.4.228.254:42879)
0810 18:47:54.574287 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:54.574294 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:54.574317 (+ 23us) server_negotiation.cc:407] Connection header received
0810 18:47:54.574541 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:54.574549 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:54.574673 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:54.574864 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:54.576644 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:54.577744 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:54.579085 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:54.579853 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:54.579996 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:54.586304 (+ 6308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:54.586315 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:54.586320 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:47:54.586409 (+ 89us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:54.590360 (+ 3951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:54.591561 (+ 1201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:54.591567 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:54.591570 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:54.591754 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:54.592496 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:54.592505 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:54.592511 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:54.592974 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:54.593599 (+ 625us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:54.593832 (+ 233us) server_negotiation.cc:299] Negotiation successful
0810 18:47:54.594124 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 18:47:54.594316 7186 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.228.252" port: 39279 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:47:54 dist-test-slave-7kzw krb5kdc[6840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387674, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.252@KRBTEST.COM for kudu/127.4.228.252@KRBTEST.COM
Aug 10 18:47:54 dist-test-slave-7kzw krb5kdc[6840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.253@KRBTEST.COM
I20260810 18:47:54.611129 7189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:54.595665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37608 (local address 127.4.228.252:39279)
0810 18:47:54.595903 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:54.595908 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:54.595924 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:47:54.595967 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:54.595970 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:54.596028 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:54.596111 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:54.597026 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:54.597797 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:54.599247 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:54.599828 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:54.599984 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:54.603926 (+ 3942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:54.603937 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:54.603942 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:47:54.603986 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:54.607684 (+ 3698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:54.608667 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:54.608672 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:54.608675 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:54.608821 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:54.609505 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:54.609511 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:54.609515 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:54.609939 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:54.610104 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:54.610445 (+ 341us) server_negotiation.cc:299] Negotiation successful
0810 18:47:54.610728 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
I20260810 18:47:54.611250 7190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:54.595298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:39279 (local address 127.0.0.1:37608)
0810 18:47:54.595630 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:54.595657 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 18:47:54.595809 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:47:54.596186 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:47:54.596191 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:47:54.596717 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:47:54.596874 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:54.596883 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:54.597937 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:54.597946 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:54.598996 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:54.599009 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:54.600026 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:54.600033 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:54.600311 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:54.601013 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:47:54.601041 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:47:54.603675 (+ 2634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:47:54.607875 (+ 4200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:47:54.607884 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:47:54.607890 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:47:54.608450 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:47:54.609223 (+ 773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:47:54.609229 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:47:54.609233 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:47:54.609385 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:47:54.610050 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:47:54.610056 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:47:54.610208 (+ 152us) client_negotiation.cc:769] Sending connection context
0810 18:47:54.610367 (+ 159us) client_negotiation.cc:240] Negotiation successful
0810 18:47:54.610881 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":153}
I20260810 18:47:54.617069 7196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:54.596592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48196 (local address 127.4.228.253:45813)
0810 18:47:54.596841 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:54.596847 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:54.596868 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:47:54.596940 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:54.596946 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:54.597071 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:54.597190 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:54.598684 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:54.599592 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:54.601598 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:54.602122 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:54.602227 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:54.608725 (+ 6498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:54.608736 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:54.608740 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:47:54.608779 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:54.612617 (+ 3838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:54.614165 (+ 1548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:54.614172 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:54.614175 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:54.614366 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:54.615051 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:54.615059 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:54.615063 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:54.615498 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:54.615827 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:54.616451 (+ 624us) server_negotiation.cc:299] Negotiation successful
0810 18:47:54.616682 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260810 18:47:54.618316 7186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 078c67ac34374f6fa3b8f7588dc421a1: Bootstrap starting.
I20260810 18:47:54.622784 7186 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 078c67ac34374f6fa3b8f7588dc421a1: Neither blocks nor log segments found. Creating new log.
I20260810 18:47:54.623805 5011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:47:54.624212 7186 log.cc:824] T 00000000000000000000000000000000 P 078c67ac34374f6fa3b8f7588dc421a1: Log is configured to *not* fsync() on all Append() calls
I20260810 18:47:54.628324 7186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 078c67ac34374f6fa3b8f7588dc421a1: No bootstrap required, opened a new log
I20260810 18:47:54.627437 6928 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "f565a2cbf6134d42969b79bce9f76b79"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51416
I20260810 18:47:54.637259 7186 raft_consensus.cc:348] T 00000000000000000000000000000000 P 078c67ac34374f6fa3b8f7588dc421a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f565a2cbf6134d42969b79bce9f76b79" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42879 } } peers { permanent_uuid: "f9b2366575374f49aee0f2ebf612670b" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 45813 } } peers { permanent_uuid: "078c67ac34374f6fa3b8f7588dc421a1" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 39279 } }
I20260810 18:47:54.637707 7186 raft_consensus.cc:729] T 00000000000000000000000000000000 P 078c67ac34374f6fa3b8f7588dc421a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 078c67ac34374f6fa3b8f7588dc421a1, State: Initialized, Role: FOLLOWER
I20260810 18:47:54.638284 7186 consensus_queue.cc:260] T 00000000000000000000000000000000 P 078c67ac34374f6fa3b8f7588dc421a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f565a2cbf6134d42969b79bce9f76b79" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42879 } } peers { permanent_uuid: "f9b2366575374f49aee0f2ebf612670b" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 45813 } } peers { permanent_uuid: "078c67ac34374f6fa3b8f7588dc421a1" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 39279 } }
I20260810 18:47:54.639564 7203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 078c67ac34374f6fa3b8f7588dc421a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f565a2cbf6134d42969b79bce9f76b79" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42879 } } peers { permanent_uuid: "f9b2366575374f49aee0f2ebf612670b" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 45813 } } peers { permanent_uuid: "078c67ac34374f6fa3b8f7588dc421a1" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 39279 } } }
I20260810 18:47:54.640205 7203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 078c67ac34374f6fa3b8f7588dc421a1 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:47:54.641001 7186 sys_catalog.cc:564] T 00000000000000000000000000000000 P 078c67ac34374f6fa3b8f7588dc421a1 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 18:47:54.652855 7223 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 078c67ac34374f6fa3b8f7588dc421a1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:47:54.653102 7223 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:47:54.649945 7096 master_runner.cc:186] Error getting master registration for 127.4.228.254:42879: OK, instance_id { permanent_uuid: "f565a2cbf6134d42969b79bce9f76b79" instance_seqno: 1786387673059878 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 18:47:54.665220 7096 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260810 18:47:54.757148 7124 tablet.cc:2369] T 00000000000000000000000000000000 P 078c67ac34374f6fa3b8f7588dc421a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:47:55.017123 7084 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.252" port: 39279 } attempt: 6
Aug 10 18:47:55 dist-test-slave-7kzw krb5kdc[6840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387673, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.253@KRBTEST.COM for kudu/127.4.228.252@KRBTEST.COM
I20260810 18:47:55.036993 6966 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.252" port: 39279 } attempt: 6
I20260810 18:47:55.039284 7225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:55.018125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:39279 (local address 127.0.0.1:37610)
0810 18:47:55.018669 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:55.018705 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 18:47:55.018857 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:47:55.019466 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:47:55.019472 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:47:55.020071 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:47:55.020317 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:55.020329 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:55.022066 (+ 1737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:55.022072 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:55.023570 (+ 1498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:55.023587 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:55.024776 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:55.024783 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:55.025156 (+ 373us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:55.026286 (+ 1130us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:47:55.026306 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:47:55.029280 (+ 2974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:47:55.035008 (+ 5728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:47:55.035018 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:47:55.035023 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:47:55.035685 (+ 662us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:47:55.036363 (+ 678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:47:55.036370 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:47:55.036374 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:47:55.036577 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:47:55.037573 (+ 996us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:47:55.037578 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:47:55.037975 (+ 397us) client_negotiation.cc:769] Sending connection context
0810 18:47:55.038151 (+ 176us) client_negotiation.cc:240] Negotiation successful
0810 18:47:55.038626 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":283,"threads_started":1}
I20260810 18:47:55.040895 7189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:55.018238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37610 (local address 127.4.228.252:39279)
0810 18:47:55.018561 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:55.018568 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:55.018810 (+ 242us) server_negotiation.cc:407] Connection header received
0810 18:47:55.018998 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:55.019004 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:55.019124 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:55.019274 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:55.020519 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:55.021880 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:55.023763 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:55.024577 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:55.024751 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:55.029599 (+ 4848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:55.029606 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:55.029611 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:47:55.029679 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:55.034766 (+ 5087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:55.036019 (+ 1253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:55.036025 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:55.036028 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:55.036209 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:55.036718 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:55.036723 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:55.036726 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:55.037415 (+ 689us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:55.037628 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:55.039905 (+ 2277us) server_negotiation.cc:299] Negotiation successful
0810 18:47:55.040275 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260810 18:47:55.047917 7084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9b2366575374f49aee0f2ebf612670b: Bootstrap starting.
I20260810 18:47:55.053176 7084 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9b2366575374f49aee0f2ebf612670b: Neither blocks nor log segments found. Creating new log.
I20260810 18:47:55.055287 7084 log.cc:824] T 00000000000000000000000000000000 P f9b2366575374f49aee0f2ebf612670b: Log is configured to *not* fsync() on all Append() calls
Aug 10 18:47:55 dist-test-slave-7kzw krb5kdc[6840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387672, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for kudu/127.4.228.252@KRBTEST.COM
I20260810 18:47:55.065672 7084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9b2366575374f49aee0f2ebf612670b: No bootstrap required, opened a new log
I20260810 18:47:55.071185 7227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:55.037889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:39279 (local address 127.0.0.1:37620)
0810 18:47:55.050811 (+ 12922us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:55.050835 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 18:47:55.051060 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:47:55.051850 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:47:55.051858 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:47:55.052744 (+ 886us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:47:55.053139 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:55.053160 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:55.055348 (+ 2188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:55.055355 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:55.057260 (+ 1905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:55.057280 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:55.058235 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:55.058241 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:55.058456 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:55.059322 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:47:55.059350 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:47:55.062696 (+ 3346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:47:55.067430 (+ 4734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:47:55.067437 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:47:55.067440 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:47:55.068141 (+ 701us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:47:55.068713 (+ 572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:47:55.068718 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:47:55.068722 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:47:55.068959 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:47:55.069878 (+ 919us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:47:55.069884 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:47:55.070082 (+ 198us) client_negotiation.cc:769] Sending connection context
0810 18:47:55.070262 (+ 180us) client_negotiation.cc:240] Negotiation successful
0810 18:47:55.070692 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12465,"thread_start_us":16572,"threads_started":1}
I20260810 18:47:55.072337 7189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:55.037810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37620 (local address 127.4.228.252:39279)
0810 18:47:55.041210 (+ 3400us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:55.041220 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:55.050975 (+ 9755us) server_negotiation.cc:407] Connection header received
0810 18:47:55.051258 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:55.051269 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:55.051397 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:55.051554 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:55.053563 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:55.055048 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:55.057435 (+ 2387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:55.058007 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:55.058178 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:55.062950 (+ 4772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:55.062956 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:55.062959 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:47:55.062997 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:55.067143 (+ 4146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:55.068345 (+ 1202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:55.068352 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:55.068355 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:55.068544 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:55.069120 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:55.069126 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:55.069129 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:55.069543 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:55.069711 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:55.071665 (+ 1954us) server_negotiation.cc:299] Negotiation successful
0810 18:47:55.071936 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3252,"thread_start_us":16933,"threads_started":1}
I20260810 18:47:55.077894 7084 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9b2366575374f49aee0f2ebf612670b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f565a2cbf6134d42969b79bce9f76b79" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42879 } } peers { permanent_uuid: "f9b2366575374f49aee0f2ebf612670b" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 45813 } } peers { permanent_uuid: "078c67ac34374f6fa3b8f7588dc421a1" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 39279 } }
I20260810 18:47:55.078356 7084 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9b2366575374f49aee0f2ebf612670b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9b2366575374f49aee0f2ebf612670b, State: Initialized, Role: FOLLOWER
I20260810 18:47:55.079151 6966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f565a2cbf6134d42969b79bce9f76b79: Bootstrap starting.
I20260810 18:47:55.078969 7084 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9b2366575374f49aee0f2ebf612670b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f565a2cbf6134d42969b79bce9f76b79" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42879 } } peers { permanent_uuid: "f9b2366575374f49aee0f2ebf612670b" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 45813 } } peers { permanent_uuid: "078c67ac34374f6fa3b8f7588dc421a1" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 39279 } }
I20260810 18:47:55.083335 7232 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9b2366575374f49aee0f2ebf612670b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f565a2cbf6134d42969b79bce9f76b79" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42879 } } peers { permanent_uuid: "f9b2366575374f49aee0f2ebf612670b" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 45813 } } peers { permanent_uuid: "078c67ac34374f6fa3b8f7588dc421a1" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 39279 } } }
I20260810 18:47:55.084291 7232 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9b2366575374f49aee0f2ebf612670b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:47:55.084568 7084 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9b2366575374f49aee0f2ebf612670b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:47:55.084957 6966 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f565a2cbf6134d42969b79bce9f76b79: Neither blocks nor log segments found. Creating new log.
I20260810 18:47:55.086604 6966 log.cc:824] T 00000000000000000000000000000000 P f565a2cbf6134d42969b79bce9f76b79: Log is configured to *not* fsync() on all Append() calls
I20260810 18:47:55.091785 6966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f565a2cbf6134d42969b79bce9f76b79: No bootstrap required, opened a new log
I20260810 18:47:55.103575 6966 raft_consensus.cc:348] T 00000000000000000000000000000000 P f565a2cbf6134d42969b79bce9f76b79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f565a2cbf6134d42969b79bce9f76b79" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42879 } } peers { permanent_uuid: "f9b2366575374f49aee0f2ebf612670b" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 45813 } } peers { permanent_uuid: "078c67ac34374f6fa3b8f7588dc421a1" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 39279 } }
I20260810 18:47:55.104202 6966 raft_consensus.cc:729] T 00000000000000000000000000000000 P f565a2cbf6134d42969b79bce9f76b79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f565a2cbf6134d42969b79bce9f76b79, State: Initialized, Role: FOLLOWER
I20260810 18:47:55.105013 6966 consensus_queue.cc:260] T 00000000000000000000000000000000 P f565a2cbf6134d42969b79bce9f76b79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f565a2cbf6134d42969b79bce9f76b79" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42879 } } peers { permanent_uuid: "f9b2366575374f49aee0f2ebf612670b" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 45813 } } peers { permanent_uuid: "078c67ac34374f6fa3b8f7588dc421a1" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 39279 } }
I20260810 18:47:55.106567 7255 sys_catalog.cc:455] T 00000000000000000000000000000000 P f565a2cbf6134d42969b79bce9f76b79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f565a2cbf6134d42969b79bce9f76b79" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42879 } } peers { permanent_uuid: "f9b2366575374f49aee0f2ebf612670b" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 45813 } } peers { permanent_uuid: "078c67ac34374f6fa3b8f7588dc421a1" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 39279 } } }
I20260810 18:47:55.107398 7255 sys_catalog.cc:458] T 00000000000000000000000000000000 P f565a2cbf6134d42969b79bce9f76b79 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:47:55.108373 6966 sys_catalog.cc:564] T 00000000000000000000000000000000 P f565a2cbf6134d42969b79bce9f76b79 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:47:55.108486 6928 raft_consensus.cc:482] T 00000000000000000000000000000000 P f565a2cbf6134d42969b79bce9f76b79 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 18:47:55.108740 6928 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f565a2cbf6134d42969b79bce9f76b79 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:47:55.112386 6928 raft_consensus.cc:504] T 00000000000000000000000000000000 P f565a2cbf6134d42969b79bce9f76b79 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f565a2cbf6134d42969b79bce9f76b79" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42879 } } peers { permanent_uuid: "f9b2366575374f49aee0f2ebf612670b" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 45813 } } peers { permanent_uuid: "078c67ac34374f6fa3b8f7588dc421a1" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 39279 } }
I20260810 18:47:55.115793 6928 leader_election.cc:288] T 00000000000000000000000000000000 P f565a2cbf6134d42969b79bce9f76b79 [CANDIDATE]: Term 1 election: Requested vote from peers f9b2366575374f49aee0f2ebf612670b (127.4.228.253:45813), 078c67ac34374f6fa3b8f7588dc421a1 (127.4.228.252:39279)
I20260810 18:47:55.116119 7036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f565a2cbf6134d42969b79bce9f76b79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f9b2366575374f49aee0f2ebf612670b"
I20260810 18:47:55.116815 7036 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9b2366575374f49aee0f2ebf612670b [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:47:55.119153 7154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f565a2cbf6134d42969b79bce9f76b79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "078c67ac34374f6fa3b8f7588dc421a1"
I20260810 18:47:55.119836 7154 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 078c67ac34374f6fa3b8f7588dc421a1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:47:55.128321 7036 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f9b2366575374f49aee0f2ebf612670b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f565a2cbf6134d42969b79bce9f76b79 in term 1.
I20260810 18:47:55.131160 7154 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 078c67ac34374f6fa3b8f7588dc421a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f565a2cbf6134d42969b79bce9f76b79 in term 1.
I20260810 18:47:55.134256 6887 leader_election.cc:302] T 00000000000000000000000000000000 P f565a2cbf6134d42969b79bce9f76b79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f565a2cbf6134d42969b79bce9f76b79, f9b2366575374f49aee0f2ebf612670b; no voters:
I20260810 18:47:55.135028 7255 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f565a2cbf6134d42969b79bce9f76b79 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:47:55.154515 7255 raft_consensus.cc:686] T 00000000000000000000000000000000 P f565a2cbf6134d42969b79bce9f76b79 [term 1 LEADER]: Becoming Leader. State: Replica: f565a2cbf6134d42969b79bce9f76b79, State: Running, Role: LEADER
I20260810 18:47:55.155627 7255 consensus_queue.cc:237] T 00000000000000000000000000000000 P f565a2cbf6134d42969b79bce9f76b79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f565a2cbf6134d42969b79bce9f76b79" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42879 } } peers { permanent_uuid: "f9b2366575374f49aee0f2ebf612670b" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 45813 } } peers { permanent_uuid: "078c67ac34374f6fa3b8f7588dc421a1" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 39279 } }
I20260810 18:47:55.161412 7265 sys_catalog.cc:455] T 00000000000000000000000000000000 P f565a2cbf6134d42969b79bce9f76b79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f565a2cbf6134d42969b79bce9f76b79. Latest consensus state: current_term: 1 leader_uuid: "f565a2cbf6134d42969b79bce9f76b79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f565a2cbf6134d42969b79bce9f76b79" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42879 } } peers { permanent_uuid: "f9b2366575374f49aee0f2ebf612670b" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 45813 } } peers { permanent_uuid: "078c67ac34374f6fa3b8f7588dc421a1" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 39279 } } }
I20260810 18:47:55.161985 7265 sys_catalog.cc:458] T 00000000000000000000000000000000 P f565a2cbf6134d42969b79bce9f76b79 [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:55.165596 7267 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260810 18:47:55.183884 7005 tablet.cc:2369] T 00000000000000000000000000000000 P f9b2366575374f49aee0f2ebf612670b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:47:55.249295 7267 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260810 18:47:55.250104 7281 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f9b2366575374f49aee0f2ebf612670b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:47:55.250274 7281 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:47:55.241572 6976 master_runner.cc:186] Error getting master registration for 127.4.228.254:42879: OK, instance_id { permanent_uuid: "f565a2cbf6134d42969b79bce9f76b79" instance_seqno: 1786387673059878 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 18:47:55.256347 6976 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 18:47:55.267329 7275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:55.172802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48198 (local address 127.4.228.253:45813)
0810 18:47:55.198150 (+ 25348us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:55.198161 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:55.203976 (+ 5815us) server_negotiation.cc:407] Connection header received
0810 18:47:55.204076 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:55.204085 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:55.204202 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:55.204338 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:55.214641 (+ 10303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:55.216968 (+ 2327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:55.233705 (+ 16737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:55.235065 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:55.243846 (+ 8781us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:55.244164 (+ 318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:55.244171 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:55.244185 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:47:55.247062 (+ 2877us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:55.258843 (+ 11781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:55.263526 (+ 4683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:55.263536 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:55.263543 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:55.263852 (+ 309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:55.264433 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:55.264439 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:55.264442 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:55.264885 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:55.265181 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:55.266482 (+ 1301us) server_negotiation.cc:299] Negotiation successful
0810 18:47:55.266802 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25092,"thread_start_us":27044,"threads_started":1}
I20260810 18:47:55.272292 7189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:55.200736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37636 (local address 127.4.228.252:39279)
0810 18:47:55.200994 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:55.201003 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:55.205262 (+ 4259us) server_negotiation.cc:407] Connection header received
0810 18:47:55.205583 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:55.205592 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:55.205706 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:55.205800 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:55.228047 (+ 22247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:55.229502 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:55.244447 (+ 14945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:55.245546 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:55.245677 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:55.261466 (+ 15789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:55.261474 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:55.261479 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:47:55.261530 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:55.265783 (+ 4253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:55.269631 (+ 3848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:55.269639 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:55.269642 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:55.269838 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:55.270287 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:55.270292 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:55.270296 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:55.270731 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:55.271023 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:55.271619 (+ 596us) server_negotiation.cc:299] Negotiation successful
0810 18:47:55.271903 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260810 18:47:55.277729 7277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:55.182270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51424 (local address 127.4.228.254:42879)
0810 18:47:55.199848 (+ 17578us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:55.199858 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:55.203329 (+ 3471us) server_negotiation.cc:407] Connection header received
0810 18:47:55.203419 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:55.203427 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:55.203520 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:55.203650 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:55.226734 (+ 23084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:55.233420 (+ 6686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:55.253562 (+ 20142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:55.254348 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:55.254496 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:55.260889 (+ 6393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:55.260899 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:55.260907 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:47:55.260961 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:55.270285 (+ 9324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:55.274109 (+ 3824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:55.274115 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:55.274119 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:55.274413 (+ 294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:55.275111 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:55.275116 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:55.275120 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:55.275498 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:55.275998 (+ 500us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:55.277017 (+ 1019us) server_negotiation.cc:299] Negotiation successful
0810 18:47:55.277230 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17365,"thread_start_us":19206,"threads_started":1}
I20260810 18:47:55.286686 7154 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 078c67ac34374f6fa3b8f7588dc421a1 [term 1 FOLLOWER]: Refusing update from remote peer f565a2cbf6134d42969b79bce9f76b79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:47:55.287971 7265 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f565a2cbf6134d42969b79bce9f76b79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "078c67ac34374f6fa3b8f7588dc421a1" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 39279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:47:55.289487 7036 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f9b2366575374f49aee0f2ebf612670b [term 1 FOLLOWER]: Refusing update from remote peer f565a2cbf6134d42969b79bce9f76b79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:47:55.290750 7265 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f565a2cbf6134d42969b79bce9f76b79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9b2366575374f49aee0f2ebf612670b" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 45813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 18:47:55.300259 6893 tablet.cc:2369] T 00000000000000000000000000000000 P f565a2cbf6134d42969b79bce9f76b79: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:47:55.302770 7232 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9b2366575374f49aee0f2ebf612670b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f565a2cbf6134d42969b79bce9f76b79. Latest consensus state: current_term: 1 leader_uuid: "f565a2cbf6134d42969b79bce9f76b79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f565a2cbf6134d42969b79bce9f76b79" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42879 } } peers { permanent_uuid: "f9b2366575374f49aee0f2ebf612670b" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 45813 } } peers { permanent_uuid: "078c67ac34374f6fa3b8f7588dc421a1" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 39279 } } }
I20260810 18:47:55.303192 7232 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9b2366575374f49aee0f2ebf612670b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:47:55.307294 7297 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 18:47:55.310068 7299 mvcc.cc:205] Tried to move back new op lower bound from 7317043917950918656 to 7317043917446553600. Current Snapshot: MvccSnapshot[applied={T|T < 7317043917950918656}]
I20260810 18:47:55.315451 7291 mvcc.cc:205] Tried to move back new op lower bound from 7317043917950918656 to 7317043917446553600. Current Snapshot: MvccSnapshot[applied={T|T < 7317043917950918656}]
I20260810 18:47:55.321640 7232 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9b2366575374f49aee0f2ebf612670b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f565a2cbf6134d42969b79bce9f76b79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f565a2cbf6134d42969b79bce9f76b79" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42879 } } peers { permanent_uuid: "f9b2366575374f49aee0f2ebf612670b" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 45813 } } peers { permanent_uuid: "078c67ac34374f6fa3b8f7588dc421a1" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 39279 } } }
I20260810 18:47:55.321959 7232 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9b2366575374f49aee0f2ebf612670b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:47:55.333338 7298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 078c67ac34374f6fa3b8f7588dc421a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f565a2cbf6134d42969b79bce9f76b79. Latest consensus state: current_term: 1 leader_uuid: "f565a2cbf6134d42969b79bce9f76b79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f565a2cbf6134d42969b79bce9f76b79" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42879 } } peers { permanent_uuid: "f9b2366575374f49aee0f2ebf612670b" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 45813 } } peers { permanent_uuid: "078c67ac34374f6fa3b8f7588dc421a1" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 39279 } } }
I20260810 18:47:55.333765 7298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 078c67ac34374f6fa3b8f7588dc421a1 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:47:55.335071 7255 sys_catalog.cc:455] T 00000000000000000000000000000000 P f565a2cbf6134d42969b79bce9f76b79 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f565a2cbf6134d42969b79bce9f76b79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f565a2cbf6134d42969b79bce9f76b79" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42879 } } peers { permanent_uuid: "f9b2366575374f49aee0f2ebf612670b" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 45813 } } peers { permanent_uuid: "078c67ac34374f6fa3b8f7588dc421a1" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 39279 } } }
I20260810 18:47:55.335448 7255 sys_catalog.cc:458] T 00000000000000000000000000000000 P f565a2cbf6134d42969b79bce9f76b79 [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:55.336133 7265 sys_catalog.cc:455] T 00000000000000000000000000000000 P f565a2cbf6134d42969b79bce9f76b79 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f565a2cbf6134d42969b79bce9f76b79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f565a2cbf6134d42969b79bce9f76b79" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42879 } } peers { permanent_uuid: "f9b2366575374f49aee0f2ebf612670b" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 45813 } } peers { permanent_uuid: "078c67ac34374f6fa3b8f7588dc421a1" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 39279 } } }
I20260810 18:47:55.336414 7265 sys_catalog.cc:458] T 00000000000000000000000000000000 P f565a2cbf6134d42969b79bce9f76b79 [sys.catalog]: This master's current role is: LEADER
I20260810 18:47:55.337435 7267 catalog_manager.cc:1347] Generated new cluster ID: 27780728e4e34b1fbde0cc23c576c29b
I20260810 18:47:55.337540 7267 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:47:55.343616 7305 mvcc.cc:205] Tried to move back new op lower bound from 7317043917950918656 to 7317043917446553600. Current Snapshot: MvccSnapshot[applied={T|T < 7317043917950918656}]
I20260810 18:47:55.349583 7298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 078c67ac34374f6fa3b8f7588dc421a1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f565a2cbf6134d42969b79bce9f76b79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f565a2cbf6134d42969b79bce9f76b79" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 42879 } } peers { permanent_uuid: "f9b2366575374f49aee0f2ebf612670b" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 45813 } } peers { permanent_uuid: "078c67ac34374f6fa3b8f7588dc421a1" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 39279 } } }
I20260810 18:47:55.350383 7298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 078c67ac34374f6fa3b8f7588dc421a1 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:47:55.379312 7267 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:47:55.381597 7267 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:47:55.406960 7267 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f565a2cbf6134d42969b79bce9f76b79: Generated new TSK 0
I20260810 18:47:55.408356 7267 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 18:47:55.534337 7275 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:47:55.519694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48202 (local address 127.4.228.253:45813)
0810 18:47:55.519923 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:55.519932 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:55.530657 (+ 10725us) server_negotiation.cc:407] Connection header received
0810 18:47:55.532533 (+ 1876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:55.532539 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:55.532639 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:55.532720 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:55.534069 (+ 1349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48202: BlockingRecv error: recv got EOF from 127.0.0.1:48202 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260810 18:47:55.538328 7189 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:47:55.531345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37642 (local address 127.4.228.252:39279)
0810 18:47:55.531546 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:55.531553 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:55.536373 (+ 4820us) server_negotiation.cc:407] Connection header received
0810 18:47:55.536642 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:55.536649 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:55.536765 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:55.536843 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:55.538059 (+ 1216us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37642: BlockingRecv error: recv got EOF from 127.0.0.1:37642 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260810 18:47:55.538728 7277 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:47:55.519213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51438 (local address 127.4.228.254:42879)
0810 18:47:55.519644 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:55.519652 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:55.536135 (+ 16483us) server_negotiation.cc:407] Connection header received
0810 18:47:55.537310 (+ 1175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:55.537317 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:55.537407 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:55.537484 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:55.538452 (+ 968us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51438: BlockingRecv error: recv got EOF from 127.0.0.1:51438 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260810 18:47:55.572690 7275 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:47:55.568541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48208 (local address 127.4.228.253:45813)
0810 18:47:55.568753 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:55.568760 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:55.570612 (+ 1852us) server_negotiation.cc:407] Connection header received
0810 18:47:55.570924 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:55.570931 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:55.571082 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:55.571212 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:55.572385 (+ 1173us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48208: BlockingRecv error: recv got EOF from 127.0.0.1:48208 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260810 18:47:55.584827 7275 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:47:55.579120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48216 (local address 127.4.228.253:45813)
0810 18:47:55.579310 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:55.579317 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:55.583128 (+ 3811us) server_negotiation.cc:407] Connection header received
0810 18:47:55.583360 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:55.583366 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:55.583479 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:55.583615 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:55.584540 (+ 925us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48216: BlockingRecv error: recv got EOF from 127.0.0.1:48216 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260810 18:47:55.593053 7275 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:47:55.590448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48222 (local address 127.4.228.253:45813)
0810 18:47:55.590697 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:55.590703 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:55.591274 (+ 571us) server_negotiation.cc:407] Connection header received
0810 18:47:55.591523 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:55.591530 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:55.591654 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:55.591781 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:55.592745 (+ 964us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48222: BlockingRecv error: recv got EOF from 127.0.0.1:48222 (error 108)
Metrics: {"server-negotiator.queue_time_us":120}
W20260810 18:47:55.604676 7275 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:47:55.601640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48230 (local address 127.4.228.253:45813)
0810 18:47:55.601836 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:55.601842 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:55.602442 (+ 600us) server_negotiation.cc:407] Connection header received
0810 18:47:55.603119 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:55.603126 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:55.603262 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:55.603427 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:55.604355 (+ 928us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48230: BlockingRecv error: recv got EOF from 127.0.0.1:48230 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260810 18:47:55.623499 7275 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:47:55.620861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48244 (local address 127.4.228.253:45813)
0810 18:47:55.621083 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:55.621091 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:55.621703 (+ 612us) server_negotiation.cc:407] Connection header received
0810 18:47:55.621930 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:55.621936 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:55.622048 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:55.622202 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:55.623237 (+ 1035us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48244: BlockingRecv error: recv got EOF from 127.0.0.1:48244 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260810 18:47:55.647353 7275 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:47:55.644344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48260 (local address 127.4.228.253:45813)
0810 18:47:55.644551 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:55.644558 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:55.645217 (+ 659us) server_negotiation.cc:407] Connection header received
0810 18:47:55.645454 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:55.645462 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:55.645626 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:55.645754 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:55.646997 (+ 1243us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48260: BlockingRecv error: recv got EOF from 127.0.0.1:48260 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 18:47:55.656272 7223 catalog_manager.cc:1259] Loaded cluster ID: 27780728e4e34b1fbde0cc23c576c29b
I20260810 18:47:55.656440 7223 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 078c67ac34374f6fa3b8f7588dc421a1: loading cluster ID for follower catalog manager: success
I20260810 18:47:55.661317 7223 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 078c67ac34374f6fa3b8f7588dc421a1: acquiring CA information for follower catalog manager: success
I20260810 18:47:55.664683 7223 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 078c67ac34374f6fa3b8f7588dc421a1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 18:47:55.695701 7275 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:47:55.692962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48268 (local address 127.4.228.253:45813)
0810 18:47:55.693236 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:55.693244 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:55.693859 (+ 615us) server_negotiation.cc:407] Connection header received
0810 18:47:55.694093 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:55.694101 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:55.694205 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:55.694346 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:55.695385 (+ 1039us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48268: BlockingRecv error: recv got EOF from 127.0.0.1:48268 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260810 18:47:55.767565 7275 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:47:55.765039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48284 (local address 127.4.228.253:45813)
0810 18:47:55.765355 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:55.765363 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:55.765765 (+ 402us) server_negotiation.cc:407] Connection header received
0810 18:47:55.766005 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:55.766014 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:55.766134 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:55.766283 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:55.767299 (+ 1016us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48284: BlockingRecv error: recv got EOF from 127.0.0.1:48284 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
W20260810 18:47:55.903478 7275 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:47:55.900563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48286 (local address 127.4.228.253:45813)
0810 18:47:55.900954 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:55.900961 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:55.901275 (+ 314us) server_negotiation.cc:407] Connection header received
0810 18:47:55.901487 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:55.901492 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:55.901623 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:55.901836 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:55.903171 (+ 1335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48286: BlockingRecv error: recv got EOF from 127.0.0.1:48286 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260810 18:47:56.171243 7275 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:47:56.169088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48288 (local address 127.4.228.253:45813)
0810 18:47:56.169369 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:56.169376 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:56.169552 (+ 176us) server_negotiation.cc:407] Connection header received
0810 18:47:56.169838 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:56.169843 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:56.169928 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:56.170061 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:56.171017 (+ 956us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48288: BlockingRecv error: recv got EOF from 127.0.0.1:48288 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
I20260810 18:47:56.252701 7281 catalog_manager.cc:1259] Loaded cluster ID: 27780728e4e34b1fbde0cc23c576c29b
I20260810 18:47:56.252889 7281 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f9b2366575374f49aee0f2ebf612670b: loading cluster ID for follower catalog manager: success
I20260810 18:47:56.256438 7281 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f9b2366575374f49aee0f2ebf612670b: acquiring CA information for follower catalog manager: success
I20260810 18:47:56.258785 7281 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f9b2366575374f49aee0f2ebf612670b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 18:47:56.695626 7409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:56.688768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48290 (local address 127.4.228.253:45813)
0810 18:47:56.689659 (+ 891us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:56.689664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:56.689681 (+ 17us) server_negotiation.cc:407] Connection header received
0810 18:47:56.689749 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:56.689753 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:56.689918 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:56.690079 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:47:56.690671 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:56.691456 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:56.692953 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:56.693420 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:56.693582 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:56.694171 (+ 589us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:47:56.694833 (+ 662us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:47:56.695020 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:56.695136 (+ 116us) server_negotiation.cc:299] Negotiation successful
0810 18:47:56.695237 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":774,"thread_start_us":671,"threads_started":1}
I20260810 18:47:56.711653 7418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:56.704277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37656 (local address 127.4.228.252:39279)
0810 18:47:56.704787 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:56.704792 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:56.704813 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:47:56.705099 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:56.705103 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:56.705274 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:56.705443 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:47:56.706395 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:56.707417 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:56.708930 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:56.709359 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:56.709533 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:56.710107 (+ 574us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:47:56.710917 (+ 810us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:47:56.711106 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:56.711212 (+ 106us) server_negotiation.cc:299] Negotiation successful
0810 18:47:56.711311 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":274,"threads_started":1}
I20260810 18:47:56.732832 7428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:56.721441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51454 (local address 127.4.228.254:42879)
0810 18:47:56.722439 (+ 998us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:56.722450 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:56.723321 (+ 871us) server_negotiation.cc:407] Connection header received
0810 18:47:56.724576 (+ 1255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:56.724591 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:56.724911 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:56.725135 (+ 224us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:47:56.726356 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:56.728035 (+ 1679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:56.730237 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:56.730781 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:56.731037 (+ 256us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:56.731549 (+ 512us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:47:56.732074 (+ 525us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:47:56.732199 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:56.732374 (+ 175us) server_negotiation.cc:299] Negotiation successful
0810 18:47:56.732465 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":818,"thread_start_us":644,"threads_started":1}
I20260810 18:47:56.733179 7409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:56.721563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48292 (local address 127.4.228.253:45813)
0810 18:47:56.721878 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:56.721889 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:56.723444 (+ 1555us) server_negotiation.cc:407] Connection header received
0810 18:47:56.723782 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:56.723791 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:56.723934 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:56.724063 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:47:56.725110 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:56.727735 (+ 2625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:56.730813 (+ 3078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:56.731515 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:56.731653 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:56.732084 (+ 431us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:47:56.732258 (+ 174us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:47:56.732365 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:56.732793 (+ 428us) server_negotiation.cc:299] Negotiation successful
0810 18:47:56.732888 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260810 18:47:56.733948 7418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:56.723767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37666 (local address 127.4.228.252:39279)
0810 18:47:56.724047 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:56.724053 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:56.725487 (+ 1434us) server_negotiation.cc:407] Connection header received
0810 18:47:56.725576 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:56.725582 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:56.725713 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:56.725818 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:47:56.727550 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:56.729079 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:56.731599 (+ 2520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:56.732254 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:56.732376 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:56.732945 (+ 569us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:47:56.733118 (+ 173us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:47:56.733210 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:56.733599 (+ 389us) server_negotiation.cc:299] Negotiation successful
0810 18:47:56.733690 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260810 18:47:56.746644 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 6861
I20260810 18:47:56.748935 6958 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:47:56.883584 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 6861
I20260810 18:47:56.922101 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 6976
I20260810 18:47:56.923544 7074 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:47:57.055392 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 6976
I20260810 18:47:57.093097 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 7096
I20260810 18:47:57.094640 7178 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:47:57.217172 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 7096
2026-08-10T18:47:57Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (6560 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:47:57 dist-test-slave-7kzw krb5kdc[7440](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:47:57 dist-test-slave-7kzw krb5kdc[7440](info): set up 2 sockets
Aug 10 18:47:57 dist-test-slave-7kzw krb5kdc[7440](info): commencing operation
krb5kdc: starting...
W20260810 18:47:58.678794 5011 mini_kdc.cc:121] Time spent starting KDC: real 1.403s user 0.002s sys 0.058s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/krb5kdc/test-user.keytab.
Aug 10 18:47:58 dist-test-slave-7kzw krb5kdc[7440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:47:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:47:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.228.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
Entry for principal kudu/127.4.228.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab.
I20260810 18:47:59.112876 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:46487
--webserver_interface=127.4.228.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:33959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:46487,127.4.228.253:35799,127.4.228.252:44099
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:47:59.547428 7456 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 18:47:59.547823 7456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:47:59.547894 7456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:47:59.547981 7456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:47:59.557214 7456 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:47:59.557358 7456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:47:59.557427 7456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:47:59.557480 7456 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:47:59.557556 7456 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:47:59.568929 7456 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:33959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:46487,127.4.228.253:35799,127.4.228.252:44099
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.254.keytab
--principal=kudu/127.4.228.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.254:46487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.228.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:47:59.570734 7456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:47:59.572167 7456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:47:59.591303 7468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:59.591573 7466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:47:59.592293 7464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:47:59.592412 7456 server_base.cc:1034] running on GCE node
I20260810 18:47:59.593781 7456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:47:59.595585 7456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:47:59.597091 7456 hybrid_clock.cc:648] HybridClock initialized: now 1786387679596935 us; error 64 us; skew 500 ppm
I20260810 18:47:59.597677 7456 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:47:59 dist-test-slave-7kzw krb5kdc[7440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387679, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:47:59.601868 7456 init.cc:377] Logged in from keytab as kudu/127.4.228.254@KRBTEST.COM (short username kudu)
I20260810 18:47:59.603750 7456 webserver.cc:460] Webserver started at http://127.4.228.254:37571/ using document root <none> and password file <none>
I20260810 18:47:59.604547 7456 fs_manager.cc:359] Metadata directory not provided
I20260810 18:47:59.604698 7456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:47:59.605065 7456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:47:59.608523 7456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-0/data/instance:
uuid: "0c387ebf88b84f8ea31cdab836f9f9e1"
format_stamp: "Formatted at 2026-08-10 18:47:59 on dist-test-slave-7kzw"
server_key: "54969ef6017d474794ecfca298d6c9a2"
server_key_iv: "d348ad6db633cc264848a625e7baff44"
server_key_version: "encryptionkey@0"
I20260810 18:47:59.609443 7456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-0/wal/instance:
uuid: "0c387ebf88b84f8ea31cdab836f9f9e1"
format_stamp: "Formatted at 2026-08-10 18:47:59 on dist-test-slave-7kzw"
server_key: "54969ef6017d474794ecfca298d6c9a2"
server_key_iv: "d348ad6db633cc264848a625e7baff44"
server_key_version: "encryptionkey@0"
I20260810 18:47:59.615579 7456 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260810 18:47:59.619899 7481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:47:59.622094 7456 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260810 18:47:59.622320 7456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-0/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-0/wal
uuid: "0c387ebf88b84f8ea31cdab836f9f9e1"
format_stamp: "Formatted at 2026-08-10 18:47:59 on dist-test-slave-7kzw"
server_key: "54969ef6017d474794ecfca298d6c9a2"
server_key_iv: "d348ad6db633cc264848a625e7baff44"
server_key_version: "encryptionkey@0"
I20260810 18:47:59.622593 7456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:47:59.649142 7456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:47:59.654954 7456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:47:59.655339 7456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:47:59.687377 7456 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.254:46487
I20260810 18:47:59.687433 7559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.254:46487 every 8 connection(s)
I20260810 18:47:59.689055 7456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-0/data/info.pb
I20260810 18:47:59.689702 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 7456
I20260810 18:47:59.690001 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-0/wal/instance
I20260810 18:47:59.690747 5011 external_mini_cluster.cc:1428] Setting key 7ebcb4dc2b576d6dbec6d688b2fce388
I20260810 18:47:59.693780 7560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:47:59.702662 7560 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46487 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:47:59 dist-test-slave-7kzw krb5kdc[7440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387679, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:47:59.733943 7564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:59.706813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:46487 (local address 127.0.0.1:33642)
0810 18:47:59.708889 (+ 2076us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:59.708935 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 18:47:59.711223 (+ 2288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:47:59.712637 (+ 1414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:47:59.712643 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:47:59.713461 (+ 818us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:47:59.713865 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:59.713885 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:59.716120 (+ 2235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:59.716131 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:59.717371 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:47:59.717387 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:47:59.718573 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:47:59.718583 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:47:59.718801 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:59.720467 (+ 1666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:47:59.720496 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:47:59.724274 (+ 3778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:47:59.728955 (+ 4681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:47:59.728967 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:47:59.728982 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:47:59.729433 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:47:59.729902 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:47:59.729907 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:47:59.729911 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:47:59.730069 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:47:59.731019 (+ 950us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:47:59.731027 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:47:59.731284 (+ 257us) client_negotiation.cc:769] Sending connection context
0810 18:47:59.731555 (+ 271us) client_negotiation.cc:240] Negotiation successful
0810 18:47:59.732213 (+ 658us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1347,"mutex_wait_us":151,"thread_start_us":1157,"threads_started":1}
I20260810 18:47:59.733952 7562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:59.706813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33642 (local address 127.4.228.254:46487)
0810 18:47:59.707923 (+ 1110us) server_negotiation.cc:206] Beginning negotiation
0810 18:47:59.707935 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:47:59.709240 (+ 1305us) server_negotiation.cc:407] Connection header received
0810 18:47:59.712028 (+ 2788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:47:59.712035 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:47:59.712367 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:47:59.712603 (+ 236us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:47:59.714420 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:59.715919 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:59.717547 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:47:59.718230 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:47:59.718517 (+ 287us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:47:59.724556 (+ 6039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:47:59.724566 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:47:59.724583 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:47:59.724628 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:47:59.728788 (+ 4160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:59.729603 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:59.729610 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:59.729612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:59.729779 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:47:59.730200 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:47:59.730206 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:47:59.730214 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:47:59.730863 (+ 649us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:47:59.731089 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0810 18:47:59.731693 (+ 604us) server_negotiation.cc:299] Negotiation successful
0810 18:47:59.732066 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":9,"server-negotiator.queue_time_us":727,"thread_start_us":478,"threads_started":1}
WARNING: no policy specified for kudu/127.4.228.253@KRBTEST.COM; defaulting to no policy
I20260810 18:47:59.737931 7560 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.228.253" port: 35799 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:47:59.739936 7564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:59.738948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:35799 (local address 127.0.0.1:47278)
0810 18:47:59.739516 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:59.739642 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.253:35799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
W20260810 18:47:59.740381 7483 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.228.253:35799: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.253@KRBTEST.COM" created.
I20260810 18:47:59.743078 7564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:59.742668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:35799 (local address 127.0.0.1:47280)
0810 18:47:59.742860 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:59.742939 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.253:35799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260810 18:47:59.743320 7560 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.253:35799: Network error: Client connection negotiation failed: client connection to 127.4.228.253:35799: connect: Connection refused (error 111)
I20260810 18:47:59.792857 7560 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.253" port: 35799 } attempt: 1
I20260810 18:47:59.794999 7564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:59.794094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:35799 (local address 127.0.0.1:47286)
0810 18:47:59.794670 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:59.794823 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.253:35799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202}
I20260810 18:47:59.796522 7564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:59.795887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:35799 (local address 127.0.0.1:47296)
0810 18:47:59.796257 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:59.796344 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.253:35799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224}
W20260810 18:47:59.797061 7560 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.253:35799: Network error: Client connection negotiation failed: client connection to 127.4.228.253:35799: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.253.keytab.
Entry for principal kudu/127.4.228.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.253.keytab.
W20260810 18:47:59.802026 5011 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.4.228.253: real 0.111s user 0.002s sys 0.040s
I20260810 18:47:59.803021 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.253:35799
--webserver_interface=127.4.228.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.253.keytab
--principal=kudu/127.4.228.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:33959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:46487,127.4.228.253:35799,127.4.228.252:44099
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:47:59.865505 7560 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.253" port: 35799 } attempt: 2
I20260810 18:47:59.867264 7564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:59.866624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:35799 (local address 127.0.0.1:47312)
0810 18:47:59.866909 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:59.867023 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.253:35799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260810 18:47:59.868598 7564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:59.868111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:35799 (local address 127.0.0.1:47314)
0810 18:47:59.868298 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:59.868409 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.253:35799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260810 18:47:59.868971 7560 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.253:35799: Network error: Client connection negotiation failed: client connection to 127.4.228.253:35799: connect: Connection refused (error 111)
I20260810 18:47:59.960347 7560 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.253" port: 35799 } attempt: 3
I20260810 18:47:59.962229 7564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:59.961538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:35799 (local address 127.0.0.1:47318)
0810 18:47:59.961870 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:59.961969 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.253:35799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163}
I20260810 18:47:59.964707 7564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:47:59.963848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:35799 (local address 127.0.0.1:47332)
0810 18:47:59.964185 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0810 18:47:59.964514 (+ 329us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.253:35799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171}
W20260810 18:47:59.965044 7560 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.253:35799: Network error: Client connection negotiation failed: client connection to 127.4.228.253:35799: connect: Connection refused (error 111)
I20260810 18:48:00.108593 7560 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.253" port: 35799 } attempt: 4
I20260810 18:48:00.111209 7564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:00.110562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:35799 (local address 127.0.0.1:47346)
0810 18:48:00.110941 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:00.111040 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.253:35799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184}
I20260810 18:48:00.112751 7564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:00.112156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:35799 (local address 127.0.0.1:47358)
0810 18:48:00.112454 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:00.112555 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.253:35799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161}
W20260810 18:48:00.113052 7560 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.253:35799: Network error: Client connection negotiation failed: client connection to 127.4.228.253:35799: connect: Connection refused (error 111)
W20260810 18:48:00.279047 7571 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 18:48:00.279806 7571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:00.279994 7571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:00.280141 7571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:00.289012 7571 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:48:00.289155 7571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:00.289223 7571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:00.289315 7571 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:48:00.289439 7571 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:48:00.301105 7571 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:33959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:46487,127.4.228.253:35799,127.4.228.252:44099
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.253.keytab
--principal=kudu/127.4.228.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.253:35799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.228.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:00.302798 7571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:00.304422 7571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:00.321552 7578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:00.323909 7583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:00.324416 7571 server_base.cc:1034] running on GCE node
W20260810 18:48:00.325302 7580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:00.328023 7571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:00.333189 7571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:00.334826 7571 hybrid_clock.cc:648] HybridClock initialized: now 1786387680334538 us; error 82 us; skew 500 ppm
I20260810 18:48:00.336086 7571 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:48:00 dist-test-slave-7kzw krb5kdc[7440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387680, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:00.342844 7571 init.cc:377] Logged in from keytab as kudu/127.4.228.253@KRBTEST.COM (short username kudu)
I20260810 18:48:00.347325 7571 webserver.cc:460] Webserver started at http://127.4.228.253:44951/ using document root <none> and password file <none>
I20260810 18:48:00.348502 7571 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:00.348721 7571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:00.349073 7571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:00.353758 7571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-1/data/instance:
uuid: "ae0bdcf5f5f24b18ad486c8b5031990e"
format_stamp: "Formatted at 2026-08-10 18:48:00 on dist-test-slave-7kzw"
server_key: "ca25cfb679996bdd7b128df1652ae041"
server_key_iv: "10a49139e4bbc52dd2f3297d71e12017"
server_key_version: "encryptionkey@0"
I20260810 18:48:00.354964 7571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-1/wal/instance:
uuid: "ae0bdcf5f5f24b18ad486c8b5031990e"
format_stamp: "Formatted at 2026-08-10 18:48:00 on dist-test-slave-7kzw"
server_key: "ca25cfb679996bdd7b128df1652ae041"
server_key_iv: "10a49139e4bbc52dd2f3297d71e12017"
server_key_version: "encryptionkey@0"
I20260810 18:48:00.363144 7571 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260810 18:48:00.369410 7596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:00.372278 7571 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260810 18:48:00.372596 7571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-1/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-1/wal
uuid: "ae0bdcf5f5f24b18ad486c8b5031990e"
format_stamp: "Formatted at 2026-08-10 18:48:00 on dist-test-slave-7kzw"
server_key: "ca25cfb679996bdd7b128df1652ae041"
server_key_iv: "10a49139e4bbc52dd2f3297d71e12017"
server_key_version: "encryptionkey@0"
I20260810 18:48:00.372906 7571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:00.397330 7571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:00.403487 7571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:00.403836 7571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:00.412326 7560 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.253" port: 35799 } attempt: 5
I20260810 18:48:00.413846 7564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:00.413164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:35799 (local address 127.0.0.1:47368)
0810 18:48:00.413540 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:00.413678 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.253:35799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187}
I20260810 18:48:00.415612 7564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:00.415055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:35799 (local address 127.0.0.1:47374)
0810 18:48:00.415363 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:00.415448 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.253:35799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196}
W20260810 18:48:00.416025 7560 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.253:35799: Network error: Client connection negotiation failed: client connection to 127.4.228.253:35799: connect: Connection refused (error 111)
I20260810 18:48:00.434370 7571 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.253:35799
I20260810 18:48:00.434304 7665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.253:35799 every 8 connection(s)
I20260810 18:48:00.436231 7571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-1/data/info.pb
I20260810 18:48:00.441461 7666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:00.442364 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 7571
I20260810 18:48:00.442627 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-1/wal/instance
I20260810 18:48:00.443274 5011 external_mini_cluster.cc:1428] Setting key e00fe59c53b341f75138a7db4f00ca6b
I20260810 18:48:00.453790 7666 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46487 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.4.228.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.228.252@KRBTEST.COM" created.
Aug 10 18:48:00 dist-test-slave-7kzw krb5kdc[7440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387680, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.253@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:48:00.509362 7672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:00.458924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:46487 (local address 127.0.0.1:33654)
0810 18:48:00.460327 (+ 1403us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:00.460404 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:00.463826 (+ 3422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:00.467299 (+ 3473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:00.467312 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:00.469106 (+ 1794us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:00.471063 (+ 1957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:00.471096 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:00.473300 (+ 2204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:00.473307 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:00.475497 (+ 2190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:00.475523 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:00.477979 (+ 2456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:00.477986 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:00.478362 (+ 376us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:00.481002 (+ 2640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:00.481037 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:00.494456 (+ 13419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:00.501075 (+ 6619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:00.501087 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:00.501111 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:00.501919 (+ 808us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:00.502600 (+ 681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:00.502604 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:00.502607 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:00.503095 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:00.504656 (+ 1561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:00.504671 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:00.505328 (+ 657us) client_negotiation.cc:769] Sending connection context
0810 18:48:00.505739 (+ 411us) client_negotiation.cc:240] Negotiation successful
0810 18:48:00.506415 (+ 676us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1027,"thread_start_us":514,"threads_started":1}
I20260810 18:48:00.509969 7670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:00.458223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33654 (local address 127.4.228.254:46487)
0810 18:48:00.458911 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:00.458918 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:00.460600 (+ 1682us) server_negotiation.cc:407] Connection header received
0810 18:48:00.464241 (+ 3641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:00.464251 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:00.464367 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:00.464502 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:00.471450 (+ 6948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:00.472844 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:00.475796 (+ 2952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:00.476978 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:00.477200 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:00.494943 (+ 17743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:00.494954 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:00.494962 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:00.495040 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:00.500278 (+ 5238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:00.502139 (+ 1861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:00.502148 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:00.502154 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:00.502371 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:00.503388 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:00.503398 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:00.503405 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:00.503923 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:00.504129 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:00.508581 (+ 4452us) server_negotiation.cc:299] Negotiation successful
0810 18:48:00.509411 (+ 830us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":312,"threads_started":1}
I20260810 18:48:00.512662 7666 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.228.253" port: 35799 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:48:00 dist-test-slave-7kzw krb5kdc[7440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387680, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.253@KRBTEST.COM for kudu/127.4.228.253@KRBTEST.COM
I20260810 18:48:00.539983 7672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:00.514961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:35799 (local address 127.0.0.1:47386)
0810 18:48:00.515250 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:00.515281 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:00.515446 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:00.516447 (+ 1001us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:00.516456 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:00.517164 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:00.517407 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:00.517419 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:00.520207 (+ 2788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:00.520211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:00.522058 (+ 1847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:00.522077 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:00.523850 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:00.523855 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:00.524028 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:00.525530 (+ 1502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:00.525553 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:00.529111 (+ 3558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:00.536097 (+ 6986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:00.536102 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:00.536106 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:00.536951 (+ 845us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:00.537835 (+ 884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:00.537845 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:00.537849 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:00.538058 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:00.539144 (+ 1086us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:00.539149 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:00.539312 (+ 163us) client_negotiation.cc:769] Sending connection context
0810 18:48:00.539440 (+ 128us) client_negotiation.cc:240] Negotiation successful
0810 18:48:00.539696 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":4}
I20260810 18:48:00.540622 7676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:00.515003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47386 (local address 127.4.228.253:35799)
0810 18:48:00.515920 (+ 917us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:00.515926 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:00.515959 (+ 33us) server_negotiation.cc:407] Connection header received
0810 18:48:00.516024 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:00.516031 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:00.516242 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:00.516383 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:00.517790 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:00.520005 (+ 2215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:00.522497 (+ 2492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:00.523643 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:00.523822 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:00.529424 (+ 5602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:00.529451 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:00.529479 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:00.529530 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:00.535834 (+ 6304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:00.537191 (+ 1357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:00.537207 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:00.537214 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:00.537406 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:00.538213 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:00.538220 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:00.538225 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:00.538879 (+ 654us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:00.539269 (+ 390us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:00.539662 (+ 393us) server_negotiation.cc:299] Negotiation successful
0810 18:48:00.540000 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":744,"thread_start_us":513,"threads_started":1}
I20260810 18:48:00.544524 7666 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.228.252" port: 44099 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:48:00.546175 7672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:00.545471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:44099 (local address 127.0.0.1:60418)
0810 18:48:00.545719 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:00.545840 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
W20260810 18:48:00.546456 7602 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.228.252:44099: connect: Connection refused (error 111)
I20260810 18:48:00.548997 7672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:00.548446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:44099 (local address 127.0.0.1:60426)
0810 18:48:00.548740 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:00.548837 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157}
W20260810 18:48:00.549276 7666 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.252:44099: Network error: Client connection negotiation failed: client connection to 127.4.228.252:44099: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.228.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.252.keytab.
Entry for principal kudu/127.4.228.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.252.keytab.
W20260810 18:48:00.560317 5011 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.4.228.252: real 0.116s user 0.003s sys 0.039s
I20260810 18:48:00.561213 5011 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
/tmp/dist-test-taskTAuB_V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.252:44099
--webserver_interface=127.4.228.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.252.keytab
--principal=kudu/127.4.228.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.228.212:33959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.228.254:46487,127.4.228.253:35799,127.4.228.252:44099
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:48:00.598691 7666 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.252" port: 44099 } attempt: 1
I20260810 18:48:00.599990 7672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:00.599482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:44099 (local address 127.0.0.1:60428)
0810 18:48:00.599732 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:00.599826 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260810 18:48:00.601358 7672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:00.600849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:44099 (local address 127.0.0.1:60430)
0810 18:48:00.601144 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:00.601235 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155}
W20260810 18:48:00.601573 7666 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.252:44099: Network error: Client connection negotiation failed: client connection to 127.4.228.252:44099: connect: Connection refused (error 111)
I20260810 18:48:00.669845 7666 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.252" port: 44099 } attempt: 2
I20260810 18:48:00.671353 7672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:00.670820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:44099 (local address 127.0.0.1:60434)
0810 18:48:00.671044 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:00.671162 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260810 18:48:00.672688 7672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:00.672253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:44099 (local address 127.0.0.1:60446)
0810 18:48:00.672452 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:00.672544 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260810 18:48:00.672940 7666 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.252:44099: Network error: Client connection negotiation failed: client connection to 127.4.228.252:44099: connect: Connection refused (error 111)
I20260810 18:48:00.764209 7666 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.252" port: 44099 } attempt: 3
I20260810 18:48:00.765538 7672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:00.765018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:44099 (local address 127.0.0.1:60456)
0810 18:48:00.765260 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:00.765369 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260810 18:48:00.766974 7672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:00.766349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:44099 (local address 127.0.0.1:60462)
0810 18:48:00.766640 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:00.766833 (+ 193us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183}
W20260810 18:48:00.767205 7666 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.252:44099: Network error: Client connection negotiation failed: client connection to 127.4.228.252:44099: connect: Connection refused (error 111)
I20260810 18:48:00.910509 7666 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.252" port: 44099 } attempt: 4
I20260810 18:48:00.911983 7672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:00.911419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:44099 (local address 127.0.0.1:60464)
0810 18:48:00.911718 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:00.911812 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260810 18:48:00.913401 7672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:00.912872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:44099 (local address 127.0.0.1:60466)
0810 18:48:00.913097 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:00.913204 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260810 18:48:00.913759 7666 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.252:44099: Network error: Client connection negotiation failed: client connection to 127.4.228.252:44099: connect: Connection refused (error 111)
I20260810 18:48:00.963389 7560 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.253" port: 35799 } attempt: 6
Aug 10 18:48:00 dist-test-slave-7kzw krb5kdc[7440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387679, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for kudu/127.4.228.253@KRBTEST.COM
I20260810 18:48:00.982012 7676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:00.964405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47396 (local address 127.4.228.253:35799)
0810 18:48:00.964634 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:00.964643 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:00.964977 (+ 334us) server_negotiation.cc:407] Connection header received
0810 18:48:00.965198 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:00.965206 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:00.965281 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:00.965482 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:00.966458 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:00.967296 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:00.968938 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:00.969398 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:00.969542 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:00.974059 (+ 4517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:00.974069 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:00.974073 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:00.974114 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:00.978262 (+ 4148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:00.979281 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:00.979290 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:00.979296 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:00.979464 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:00.980410 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:00.980420 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:00.980426 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:00.980797 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:00.980978 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:00.981410 (+ 432us) server_negotiation.cc:299] Negotiation successful
0810 18:48:00.981640 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 18:48:00.982085 7681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:00.964307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:35799 (local address 127.0.0.1:47396)
0810 18:48:00.964855 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:00.964872 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:00.965052 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:00.965518 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:00.965523 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:00.966100 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:00.966286 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:00.966297 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:00.967442 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:00.967448 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:00.968659 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:00.968674 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:00.969590 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:00.969596 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:00.969793 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:00.970822 (+ 1029us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:00.970843 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:00.973778 (+ 2935us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:00.978490 (+ 4712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:00.978498 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:00.978503 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:00.979121 (+ 618us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:00.979611 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:00.979617 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:00.979621 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:00.979903 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:00.980944 (+ 1041us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:00.980950 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:00.981122 (+ 172us) client_negotiation.cc:769] Sending connection context
0810 18:48:00.981287 (+ 165us) client_negotiation.cc:240] Negotiation successful
0810 18:48:00.981620 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":303,"threads_started":1}
I20260810 18:48:00.983785 7560 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.228.252" port: 44099 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:48:00.985380 7681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:00.984858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:44099 (local address 127.0.0.1:60482)
0810 18:48:00.985095 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:00.985236 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260810 18:48:00.986963 7681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:00.986605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:44099 (local address 127.0.0.1:60490)
0810 18:48:00.986761 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:00.986830 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 18:48:00.987210 7560 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.252:44099: Network error: Client connection negotiation failed: client connection to 127.4.228.252:44099: connect: Connection refused (error 111)
W20260810 18:48:01.024806 7680 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 18:48:01.025309 7680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:48:01.025451 7680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:48:01.025602 7680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:48:01.034989 7680 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:48:01.035152 7680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:48:01.035668 7680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:48:01.035840 7680 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:48:01.035976 7680 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:48:01.039489 7560 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.252" port: 44099 } attempt: 1
I20260810 18:48:01.040911 7681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:01.040315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:44099 (local address 127.0.0.1:60500)
0810 18:48:01.040588 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:01.040720 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260810 18:48:01.042248 7681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:01.041766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:44099 (local address 127.0.0.1:60502)
0810 18:48:01.041948 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:01.042065 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260810 18:48:01.042626 7560 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.252:44099: Network error: Client connection negotiation failed: client connection to 127.4.228.252:44099: connect: Connection refused (error 111)
I20260810 18:48:01.053615 7680 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.228.212:33959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.228.254:46487,127.4.228.253:35799,127.4.228.252:44099
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/krb5kdc/kudu_127.4.228.252.keytab
--principal=kudu/127.4.228.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.228.252:44099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.228.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 17:48:56 UTC on ccycloud.quasar-rratoj.root.comops.site
build id 4578680
ASAN enabled
I20260810 18:48:01.055503 7680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:48:01.057482 7680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:48:01.074968 7690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:01.074899 7698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:48:01.074932 7695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:48:01.077272 7680 server_base.cc:1034] running on GCE node
I20260810 18:48:01.079437 7680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:48:01.081454 7680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:48:01.084915 7680 hybrid_clock.cc:648] HybridClock initialized: now 1786387681084587 us; error 425 us; skew 500 ppm
I20260810 18:48:01.085407 7680 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 18:48:01 dist-test-slave-7kzw krb5kdc[7440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387681, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:48:01.090122 7680 init.cc:377] Logged in from keytab as kudu/127.4.228.252@KRBTEST.COM (short username kudu)
I20260810 18:48:01.092253 7680 webserver.cc:460] Webserver started at http://127.4.228.252:46115/ using document root <none> and password file <none>
I20260810 18:48:01.093058 7680 fs_manager.cc:359] Metadata directory not provided
I20260810 18:48:01.093175 7680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:48:01.094005 7680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:48:01.097970 7680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-2/data/instance:
uuid: "abb4ea345a894f9685874057211a0697"
format_stamp: "Formatted at 2026-08-10 18:48:01 on dist-test-slave-7kzw"
server_key: "e219bd1b6182e8cd6538f870006ba0ab"
server_key_iv: "89fc0d06052897f50a7600e46344097a"
server_key_version: "encryptionkey@0"
I20260810 18:48:01.099350 7680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-2/wal/instance:
uuid: "abb4ea345a894f9685874057211a0697"
format_stamp: "Formatted at 2026-08-10 18:48:01 on dist-test-slave-7kzw"
server_key: "e219bd1b6182e8cd6538f870006ba0ab"
server_key_iv: "89fc0d06052897f50a7600e46344097a"
server_key_version: "encryptionkey@0"
I20260810 18:48:01.106324 7680 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260810 18:48:01.111539 7705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:48:01.113456 7680 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:48:01.113832 7680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-2/data,/tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-2/wal
uuid: "abb4ea345a894f9685874057211a0697"
format_stamp: "Formatted at 2026-08-10 18:48:01 on dist-test-slave-7kzw"
server_key: "e219bd1b6182e8cd6538f870006ba0ab"
server_key_iv: "89fc0d06052897f50a7600e46344097a"
server_key_version: "encryptionkey@0"
I20260810 18:48:01.114130 7680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:48:01.116899 7560 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.252" port: 44099 } attempt: 2
I20260810 18:48:01.118160 7681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:01.117580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:44099 (local address 127.0.0.1:60504)
0810 18:48:01.117831 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:01.117940 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260810 18:48:01.120288 7681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:01.119057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:44099 (local address 127.0.0.1:60512)
0810 18:48:01.119460 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:01.119590 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.228.252:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260810 18:48:01.121040 7560 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.228.252:44099: Network error: Client connection negotiation failed: client connection to 127.4.228.252:44099: connect: Connection refused (error 111)
I20260810 18:48:01.139312 7680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:48:01.146346 7680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:48:01.146685 7680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:48:01.175587 7680 rpc_server.cc:307] RPC server started. Bound to: 127.4.228.252:44099
I20260810 18:48:01.175619 7779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.228.252:44099 every 8 connection(s)
I20260810 18:48:01.177249 7680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-2/data/info.pb
I20260810 18:48:01.183194 5011 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu as pid 7680
I20260810 18:48:01.183429 5011 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTAuB_V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387267874959-5011-0/minicluster-data/master-2/wal/instance
I20260810 18:48:01.184441 5011 external_mini_cluster.cc:1428] Setting key c83397314ba8c2e74f12d25a2a418a81
I20260810 18:48:01.187031 7780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:48:01.198295 7780 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46487 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:48:01 dist-test-slave-7kzw krb5kdc[7440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.252@KRBTEST.COM
I20260810 18:48:01.213068 7666 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.252" port: 44099 } attempt: 5
Aug 10 18:48:01 dist-test-slave-7kzw krb5kdc[7440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387681, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.252@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:48:01.219429 7785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:01.188097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60526 (local address 127.4.228.252:44099)
0810 18:48:01.189111 (+ 1014us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:01.189129 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:01.189191 (+ 62us) server_negotiation.cc:407] Connection header received
0810 18:48:01.191399 (+ 2208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:01.191433 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:01.192066 (+ 633us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:01.192558 (+ 492us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:01.194630 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:01.196823 (+ 2193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:01.198762 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:01.200239 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:01.200475 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:01.207253 (+ 6778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:01.207268 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:01.207300 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:01.207356 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:01.212991 (+ 5635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:01.214091 (+ 1100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:01.214103 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:01.214114 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:01.214397 (+ 283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:01.215247 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:01.215257 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:01.215260 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:01.215912 (+ 652us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:01.216375 (+ 463us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:01.216812 (+ 437us) server_negotiation.cc:299] Negotiation successful
0810 18:48:01.217543 (+ 731us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":742,"thread_start_us":371,"threads_started":1}
Aug 10 18:48:01 dist-test-slave-7kzw krb5kdc[7440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387680, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.253@KRBTEST.COM for kudu/127.4.228.252@KRBTEST.COM
I20260810 18:48:01.230140 7788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:01.202770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33664 (local address 127.4.228.254:46487)
0810 18:48:01.204235 (+ 1465us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:01.204244 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:01.204273 (+ 29us) server_negotiation.cc:407] Connection header received
0810 18:48:01.204902 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:01.204907 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:01.204983 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:01.205209 (+ 226us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:01.208956 (+ 3747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:01.209987 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:01.212942 (+ 2955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:01.213595 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:01.213768 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:01.219359 (+ 5591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:01.219376 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:01.219384 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:01.219465 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:01.225255 (+ 5790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:01.226383 (+ 1128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:01.226389 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:01.226395 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:01.226767 (+ 372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:01.227649 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:01.227655 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:01.227658 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:01.228249 (+ 591us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:01.228586 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:01.229347 (+ 761us) server_negotiation.cc:299] Negotiation successful
0810 18:48:01.229704 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1302,"thread_start_us":1167,"threads_started":1}
I20260810 18:48:01.230172 7786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:01.202297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.254:46487 (local address 127.0.0.1:33664)
0810 18:48:01.203905 (+ 1608us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:01.203969 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:01.204662 (+ 693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:01.205303 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:01.205315 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:01.207319 (+ 2004us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:01.208161 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:01.208179 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:01.210318 (+ 2139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:01.210323 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:01.212476 (+ 2153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:01.212526 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:01.213834 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:01.213842 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:01.214050 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:01.215654 (+ 1604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:01.215686 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:01.218934 (+ 3248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:01.225437 (+ 6503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:01.225459 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:01.225465 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:01.226187 (+ 722us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:01.226952 (+ 765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:01.226959 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:01.226966 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:01.227405 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:01.228683 (+ 1278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:01.228690 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:01.228981 (+ 291us) client_negotiation.cc:769] Sending connection context
0810 18:48:01.229192 (+ 211us) client_negotiation.cc:240] Negotiation successful
0810 18:48:01.229704 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":962,"thread_start_us":624,"threads_started":1}
I20260810 18:48:01.232113 7780 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.228.253" port: 35799 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:48:01.234355 7560 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.228.252" port: 44099 } attempt: 3
I20260810 18:48:01.239818 7672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:01.213973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:44099 (local address 127.0.0.1:60536)
0810 18:48:01.214289 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:01.214312 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:01.214452 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:01.216392 (+ 1940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:01.216399 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:01.217013 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:01.217273 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:01.217307 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:01.219062 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:01.219069 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:01.220952 (+ 1883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:01.220978 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:01.222382 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:01.222392 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:01.222849 (+ 457us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:01.224979 (+ 2130us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:01.225060 (+ 81us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:01.230588 (+ 5528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:01.234942 (+ 4354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:01.234947 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:01.234951 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:01.235688 (+ 737us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:01.237053 (+ 1365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:01.237061 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:01.237067 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:01.237271 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:01.238230 (+ 959us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:01.238238 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:01.238436 (+ 198us) client_negotiation.cc:769] Sending connection context
0810 18:48:01.238717 (+ 281us) client_negotiation.cc:240] Negotiation successful
0810 18:48:01.239115 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181}
Aug 10 18:48:01 dist-test-slave-7kzw krb5kdc[7440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.254@KRBTEST.COM
I20260810 18:48:01.241001 7789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:01.214214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60536 (local address 127.4.228.252:44099)
0810 18:48:01.215879 (+ 1665us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:01.215885 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:01.215909 (+ 24us) server_negotiation.cc:407] Connection header received
0810 18:48:01.215957 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:01.215963 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:01.216044 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:01.217146 (+ 1102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:01.217481 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:01.218894 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:01.221238 (+ 2344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:01.221930 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:01.222131 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:01.230857 (+ 8726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:01.230862 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:01.230865 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:01.230896 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:01.234770 (+ 3874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:01.236579 (+ 1809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:01.236589 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:01.236599 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:01.236860 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:01.237444 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:01.237451 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:01.237457 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:01.237957 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:01.238142 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:01.240200 (+ 2058us) server_negotiation.cc:299] Negotiation successful
0810 18:48:01.240464 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1511,"thread_start_us":536,"threads_started":1}
Aug 10 18:48:01 dist-test-slave-7kzw krb5kdc[7440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387681, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.252@KRBTEST.COM for kudu/127.4.228.253@KRBTEST.COM
Aug 10 18:48:01 dist-test-slave-7kzw krb5kdc[7440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387679, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.254@KRBTEST.COM for kudu/127.4.228.252@KRBTEST.COM
I20260810 18:48:01.250535 7666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae0bdcf5f5f24b18ad486c8b5031990e: Bootstrap starting.
I20260810 18:48:01.252674 7790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:01.226366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33670 (local address 127.4.228.254:46487)
0810 18:48:01.227768 (+ 1402us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:01.227774 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:01.227802 (+ 28us) server_negotiation.cc:407] Connection header received
0810 18:48:01.227855 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:01.227872 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:01.227953 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:01.228072 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:01.230438 (+ 2366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:01.232164 (+ 1726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:01.234047 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:01.234807 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:01.242255 (+ 7448us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:01.242553 (+ 298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:01.242567 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:01.242575 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:01.242675 (+ 100us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:01.247483 (+ 4808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:01.248662 (+ 1179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:01.248668 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:01.248673 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:01.248894 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:01.249705 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:01.249710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:01.249714 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:01.250168 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:01.251803 (+ 1635us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:01.251942 (+ 139us) server_negotiation.cc:299] Negotiation successful
0810 18:48:01.252204 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1227,"thread_start_us":1024,"threads_started":1}
I20260810 18:48:01.253902 7676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:01.233312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47402 (local address 127.4.228.253:35799)
0810 18:48:01.233525 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:01.233532 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:01.233553 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:48:01.233610 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:01.233617 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:01.233700 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:01.233814 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:01.234956 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:01.236252 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:01.238145 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:01.239077 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:01.239189 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:01.245161 (+ 5972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:01.245170 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:01.245175 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:01.245256 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:01.249576 (+ 4320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:01.250702 (+ 1126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:01.250708 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:01.250712 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:01.251171 (+ 459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:01.251764 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:01.251770 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:01.251774 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:01.252372 (+ 598us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:01.252519 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:01.253149 (+ 630us) server_negotiation.cc:299] Negotiation successful
0810 18:48:01.253466 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 18:48:01.254709 7681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:01.235037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:44099 (local address 127.0.0.1:60544)
0810 18:48:01.235381 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:01.235400 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:01.235502 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:01.236259 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:01.236264 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:01.236812 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:01.236991 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:01.237006 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:01.238408 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:01.238414 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:01.240001 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:01.240018 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:01.241715 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:01.241724 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:01.241952 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:01.242765 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:01.242796 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:01.246817 (+ 4021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:01.251437 (+ 4620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:01.251442 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:01.251446 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:01.252025 (+ 579us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:01.252485 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:01.252490 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:01.252493 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:01.252703 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:01.253437 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:01.253442 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:01.253825 (+ 383us) client_negotiation.cc:769] Sending connection context
0810 18:48:01.254015 (+ 190us) client_negotiation.cc:240] Negotiation successful
0810 18:48:01.254313 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220}
I20260810 18:48:01.255350 7786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:01.233068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.253:35799 (local address 127.0.0.1:47402)
0810 18:48:01.233370 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:01.233389 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:01.233489 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:01.233869 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:01.233874 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:01.234529 (+ 655us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:01.234764 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:01.234779 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:01.236477 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:01.236484 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:01.237976 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:01.237997 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:01.239501 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:01.239507 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:01.239694 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:01.240601 (+ 907us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:01.240632 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:01.244897 (+ 4265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:01.249792 (+ 4895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:01.249797 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:01.249800 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:01.250539 (+ 739us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:01.251393 (+ 854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:01.251399 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:01.251402 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:01.251605 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:01.252617 (+ 1012us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:01.252621 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:01.252815 (+ 194us) client_negotiation.cc:769] Sending connection context
0810 18:48:01.254320 (+ 1505us) client_negotiation.cc:240] Negotiation successful
0810 18:48:01.254666 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170}
I20260810 18:48:01.255661 7785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:01.235692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60544 (local address 127.4.228.252:44099)
0810 18:48:01.235862 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:01.235867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:01.235884 (+ 17us) server_negotiation.cc:407] Connection header received
0810 18:48:01.235927 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:01.235932 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:01.235998 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:01.236080 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:01.237169 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:01.238263 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:01.240579 (+ 2316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:01.241501 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:01.241651 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:01.247080 (+ 5429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:01.247089 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:01.247095 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:01.247139 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:01.251257 (+ 4118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:01.252174 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:01.252179 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:01.252182 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:01.252350 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:01.252859 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:01.252863 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:01.252866 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:01.253214 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:01.253371 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:01.255025 (+ 1654us) server_negotiation.cc:299] Negotiation successful
0810 18:48:01.255286 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 18:48:01.256871 7780 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.228.252" port: 44099 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:48:01.259853 7666 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae0bdcf5f5f24b18ad486c8b5031990e: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:01.261376 7560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c387ebf88b84f8ea31cdab836f9f9e1: Bootstrap starting.
I20260810 18:48:01.261601 7666 log.cc:824] T 00000000000000000000000000000000 P ae0bdcf5f5f24b18ad486c8b5031990e: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:01.267112 7666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae0bdcf5f5f24b18ad486c8b5031990e: No bootstrap required, opened a new log
I20260810 18:48:01.267078 7560 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c387ebf88b84f8ea31cdab836f9f9e1: Neither blocks nor log segments found. Creating new log.
Aug 10 18:48:01 dist-test-slave-7kzw krb5kdc[7440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.228.253@KRBTEST.COM
I20260810 18:48:01.268646 7560 log.cc:824] T 00000000000000000000000000000000 P 0c387ebf88b84f8ea31cdab836f9f9e1: Log is configured to *not* fsync() on all Append() calls
Aug 10 18:48:01 dist-test-slave-7kzw krb5kdc[7440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387681, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.228.252@KRBTEST.COM for kudu/127.4.228.252@KRBTEST.COM
I20260810 18:48:01.279969 7560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c387ebf88b84f8ea31cdab836f9f9e1: No bootstrap required, opened a new log
I20260810 18:48:01.281561 7676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:01.255404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47418 (local address 127.4.228.253:35799)
0810 18:48:01.255580 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:01.255585 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:01.255612 (+ 27us) server_negotiation.cc:407] Connection header received
0810 18:48:01.255665 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:01.255670 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:01.255745 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:01.255844 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:01.258744 (+ 2900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:01.260106 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:01.261951 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:01.262681 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:01.262826 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:01.269994 (+ 7168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:01.270003 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:01.270008 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:01.270075 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:01.275694 (+ 5619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:01.277015 (+ 1321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:01.277029 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:01.277032 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:01.277283 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:01.277901 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:01.277909 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:01.277912 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:01.278449 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:01.278781 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:01.280721 (+ 1940us) server_negotiation.cc:299] Negotiation successful
0810 18:48:01.281051 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 18:48:01.286725 7785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:01.257876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60558 (local address 127.4.228.252:44099)
0810 18:48:01.258302 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:01.258308 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:01.266863 (+ 8555us) server_negotiation.cc:407] Connection header received
0810 18:48:01.267145 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:01.267153 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:01.267251 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:01.267380 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:01.268914 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:01.270126 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:01.272149 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:01.273018 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:01.273156 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:01.278593 (+ 5437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:01.278600 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:01.278605 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:01.278653 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:01.283307 (+ 4654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:01.284322 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:01.284328 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:01.284331 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:01.284488 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:01.284856 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:01.284861 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:01.284865 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:01.285228 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:01.285445 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:01.285876 (+ 431us) server_negotiation.cc:299] Negotiation successful
0810 18:48:01.286323 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200}
I20260810 18:48:01.286120 7666 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae0bdcf5f5f24b18ad486c8b5031990e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c387ebf88b84f8ea31cdab836f9f9e1" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46487 } } peers { permanent_uuid: "ae0bdcf5f5f24b18ad486c8b5031990e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 35799 } } peers { permanent_uuid: "abb4ea345a894f9685874057211a0697" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 44099 } }
I20260810 18:48:01.287034 7666 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae0bdcf5f5f24b18ad486c8b5031990e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae0bdcf5f5f24b18ad486c8b5031990e, State: Initialized, Role: FOLLOWER
I20260810 18:48:01.288074 7786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:01.257900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.228.252:44099 (local address 127.0.0.1:60558)
0810 18:48:01.266749 (+ 8849us) negotiation.cc:107] Waiting for socket to connect
0810 18:48:01.266777 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 18:48:01.266980 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:48:01.267757 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:48:01.267763 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:48:01.268477 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:48:01.268733 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:01.268748 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:01.270315 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:01.270322 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:01.271975 (+ 1653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:48:01.271995 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:48:01.273490 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:48:01.273499 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:48:01.273750 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:01.274446 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:48:01.274500 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:48:01.278263 (+ 3763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:48:01.283535 (+ 5272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:01.283541 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:01.283545 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:01.284191 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:01.284601 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:48:01.284605 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:48:01.284607 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:48:01.284716 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:48:01.285369 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:48:01.285374 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:48:01.285586 (+ 212us) client_negotiation.cc:769] Sending connection context
0810 18:48:01.287114 (+ 1528us) client_negotiation.cc:240] Negotiation successful
0810 18:48:01.287660 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8655}
I20260810 18:48:01.288091 7666 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae0bdcf5f5f24b18ad486c8b5031990e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c387ebf88b84f8ea31cdab836f9f9e1" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46487 } } peers { permanent_uuid: "ae0bdcf5f5f24b18ad486c8b5031990e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 35799 } } peers { permanent_uuid: "abb4ea345a894f9685874057211a0697" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 44099 } }
I20260810 18:48:01.290223 5011 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:48:01.290043 7793 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae0bdcf5f5f24b18ad486c8b5031990e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c387ebf88b84f8ea31cdab836f9f9e1" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46487 } } peers { permanent_uuid: "ae0bdcf5f5f24b18ad486c8b5031990e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 35799 } } peers { permanent_uuid: "abb4ea345a894f9685874057211a0697" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 44099 } } }
I20260810 18:48:01.290509 7793 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae0bdcf5f5f24b18ad486c8b5031990e [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:48:01.292865 7520 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "0c387ebf88b84f8ea31cdab836f9f9e1"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33670
I20260810 18:48:01.297168 7780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abb4ea345a894f9685874057211a0697: Bootstrap starting.
I20260810 18:48:01.298080 7560 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c387ebf88b84f8ea31cdab836f9f9e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c387ebf88b84f8ea31cdab836f9f9e1" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46487 } } peers { permanent_uuid: "ae0bdcf5f5f24b18ad486c8b5031990e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 35799 } } peers { permanent_uuid: "abb4ea345a894f9685874057211a0697" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 44099 } }
I20260810 18:48:01.298653 7560 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c387ebf88b84f8ea31cdab836f9f9e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c387ebf88b84f8ea31cdab836f9f9e1, State: Initialized, Role: FOLLOWER
I20260810 18:48:01.299423 7560 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c387ebf88b84f8ea31cdab836f9f9e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c387ebf88b84f8ea31cdab836f9f9e1" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46487 } } peers { permanent_uuid: "ae0bdcf5f5f24b18ad486c8b5031990e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 35799 } } peers { permanent_uuid: "abb4ea345a894f9685874057211a0697" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 44099 } }
I20260810 18:48:01.299947 7666 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae0bdcf5f5f24b18ad486c8b5031990e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:48:01.301146 7794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c387ebf88b84f8ea31cdab836f9f9e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c387ebf88b84f8ea31cdab836f9f9e1" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46487 } } peers { permanent_uuid: "ae0bdcf5f5f24b18ad486c8b5031990e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 35799 } } peers { permanent_uuid: "abb4ea345a894f9685874057211a0697" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 44099 } } }
I20260810 18:48:01.302013 7780 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P abb4ea345a894f9685874057211a0697: Neither blocks nor log segments found. Creating new log.
I20260810 18:48:01.302245 7560 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c387ebf88b84f8ea31cdab836f9f9e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:48:01.302333 7520 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c387ebf88b84f8ea31cdab836f9f9e1 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 18:48:01.302578 7794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c387ebf88b84f8ea31cdab836f9f9e1 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:48:01.302770 7520 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c387ebf88b84f8ea31cdab836f9f9e1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:01.303519 7780 log.cc:824] T 00000000000000000000000000000000 P abb4ea345a894f9685874057211a0697: Log is configured to *not* fsync() on all Append() calls
I20260810 18:48:01.307061 7520 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c387ebf88b84f8ea31cdab836f9f9e1 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c387ebf88b84f8ea31cdab836f9f9e1" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46487 } } peers { permanent_uuid: "ae0bdcf5f5f24b18ad486c8b5031990e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 35799 } } peers { permanent_uuid: "abb4ea345a894f9685874057211a0697" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 44099 } }
I20260810 18:48:01.307781 7780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abb4ea345a894f9685874057211a0697: No bootstrap required, opened a new log
I20260810 18:48:01.310122 7636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0c387ebf88b84f8ea31cdab836f9f9e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ae0bdcf5f5f24b18ad486c8b5031990e"
I20260810 18:48:01.310875 7636 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae0bdcf5f5f24b18ad486c8b5031990e [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:01.315308 7636 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ae0bdcf5f5f24b18ad486c8b5031990e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c387ebf88b84f8ea31cdab836f9f9e1 in term 1.
I20260810 18:48:01.316277 7483 leader_election.cc:302] T 00000000000000000000000000000000 P 0c387ebf88b84f8ea31cdab836f9f9e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c387ebf88b84f8ea31cdab836f9f9e1, ae0bdcf5f5f24b18ad486c8b5031990e; no voters:
I20260810 18:48:01.316929 7794 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c387ebf88b84f8ea31cdab836f9f9e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:48:01.321683 7780 raft_consensus.cc:348] T 00000000000000000000000000000000 P abb4ea345a894f9685874057211a0697 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c387ebf88b84f8ea31cdab836f9f9e1" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46487 } } peers { permanent_uuid: "ae0bdcf5f5f24b18ad486c8b5031990e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 35799 } } peers { permanent_uuid: "abb4ea345a894f9685874057211a0697" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 44099 } }
I20260810 18:48:01.322295 7780 raft_consensus.cc:729] T 00000000000000000000000000000000 P abb4ea345a894f9685874057211a0697 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abb4ea345a894f9685874057211a0697, State: Initialized, Role: FOLLOWER
I20260810 18:48:01.322233 7520 leader_election.cc:288] T 00000000000000000000000000000000 P 0c387ebf88b84f8ea31cdab836f9f9e1 [CANDIDATE]: Term 1 election: Requested vote from peers ae0bdcf5f5f24b18ad486c8b5031990e (127.4.228.253:35799), abb4ea345a894f9685874057211a0697 (127.4.228.252:44099)
I20260810 18:48:01.322958 7780 consensus_queue.cc:260] T 00000000000000000000000000000000 P abb4ea345a894f9685874057211a0697 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c387ebf88b84f8ea31cdab836f9f9e1" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46487 } } peers { permanent_uuid: "ae0bdcf5f5f24b18ad486c8b5031990e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 35799 } } peers { permanent_uuid: "abb4ea345a894f9685874057211a0697" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 44099 } }
I20260810 18:48:01.327945 7801 sys_catalog.cc:455] T 00000000000000000000000000000000 P abb4ea345a894f9685874057211a0697 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c387ebf88b84f8ea31cdab836f9f9e1" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46487 } } peers { permanent_uuid: "ae0bdcf5f5f24b18ad486c8b5031990e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 35799 } } peers { permanent_uuid: "abb4ea345a894f9685874057211a0697" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 44099 } } }
I20260810 18:48:01.328526 7801 sys_catalog.cc:458] T 00000000000000000000000000000000 P abb4ea345a894f9685874057211a0697 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:48:01.331310 7780 sys_catalog.cc:564] T 00000000000000000000000000000000 P abb4ea345a894f9685874057211a0697 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:48:01.331372 7745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0c387ebf88b84f8ea31cdab836f9f9e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "abb4ea345a894f9685874057211a0697"
I20260810 18:48:01.331934 7745 raft_consensus.cc:3037] T 00000000000000000000000000000000 P abb4ea345a894f9685874057211a0697 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:48:01.336908 7745 raft_consensus.cc:2454] T 00000000000000000000000000000000 P abb4ea345a894f9685874057211a0697 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c387ebf88b84f8ea31cdab836f9f9e1 in term 1.
I20260810 18:48:01.352934 7794 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c387ebf88b84f8ea31cdab836f9f9e1 [term 1 LEADER]: Becoming Leader. State: Replica: 0c387ebf88b84f8ea31cdab836f9f9e1, State: Running, Role: LEADER
I20260810 18:48:01.354387 7794 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c387ebf88b84f8ea31cdab836f9f9e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c387ebf88b84f8ea31cdab836f9f9e1" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46487 } } peers { permanent_uuid: "ae0bdcf5f5f24b18ad486c8b5031990e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 35799 } } peers { permanent_uuid: "abb4ea345a894f9685874057211a0697" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 44099 } }
I20260810 18:48:01.371865 7800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c387ebf88b84f8ea31cdab836f9f9e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c387ebf88b84f8ea31cdab836f9f9e1. Latest consensus state: current_term: 1 leader_uuid: "0c387ebf88b84f8ea31cdab836f9f9e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c387ebf88b84f8ea31cdab836f9f9e1" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46487 } } peers { permanent_uuid: "ae0bdcf5f5f24b18ad486c8b5031990e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 35799 } } peers { permanent_uuid: "abb4ea345a894f9685874057211a0697" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 44099 } } }
I20260810 18:48:01.372552 7800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c387ebf88b84f8ea31cdab836f9f9e1 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:01.380921 7839 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:48:01.395367 7839 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260810 18:48:01.402186 7718 tablet.cc:2369] T 00000000000000000000000000000000 P abb4ea345a894f9685874057211a0697: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:48:01.408270 7605 tablet.cc:2369] T 00000000000000000000000000000000 P ae0bdcf5f5f24b18ad486c8b5031990e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:48:01.409865 7491 tablet.cc:2369] T 00000000000000000000000000000000 P 0c387ebf88b84f8ea31cdab836f9f9e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:48:01.411723 7676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:01.359271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47424 (local address 127.4.228.253:35799)
0810 18:48:01.359544 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:01.359554 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:01.361179 (+ 1625us) server_negotiation.cc:407] Connection header received
0810 18:48:01.364381 (+ 3202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:01.364391 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:01.364539 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:01.364645 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:01.375246 (+ 10601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:01.377391 (+ 2145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:01.387985 (+ 10594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:01.388764 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:01.388890 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:01.397531 (+ 8641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:01.397543 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:01.397552 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:01.397614 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:01.406143 (+ 8529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:01.407615 (+ 1472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:01.407641 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:01.407649 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:01.407912 (+ 263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:01.408539 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:01.408545 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:01.408548 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:01.409159 (+ 611us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:01.409513 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:01.410979 (+ 1466us) server_negotiation.cc:299] Negotiation successful
0810 18:48:01.411267 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260810 18:48:01.421921 7790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:01.361540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33676 (local address 127.4.228.254:46487)
0810 18:48:01.361856 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:01.361866 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:01.362407 (+ 541us) server_negotiation.cc:407] Connection header received
0810 18:48:01.367298 (+ 4891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:01.367308 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:01.367421 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:01.367532 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:01.374024 (+ 6492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:01.375999 (+ 1975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:01.384412 (+ 8413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:01.385585 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:01.385749 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:01.396338 (+ 10589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:01.396352 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:01.396360 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:01.396440 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:01.410902 (+ 14462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:01.412195 (+ 1293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:01.412206 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:01.412214 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:01.412434 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:01.413305 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:01.413314 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:01.413320 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:01.413813 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:01.417636 (+ 3823us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:01.420721 (+ 3085us) server_negotiation.cc:299] Negotiation successful
0810 18:48:01.421102 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 18:48:01.425465 7785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:01.360924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60572 (local address 127.4.228.252:44099)
0810 18:48:01.361112 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:01.361120 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:01.367092 (+ 5972us) server_negotiation.cc:407] Connection header received
0810 18:48:01.369811 (+ 2719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:01.369817 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:01.369918 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:01.370002 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:01.381612 (+ 11610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:01.382735 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:01.390204 (+ 7469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:01.391067 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:01.391208 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:01.411462 (+ 20254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:48:01.411470 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:48:01.411475 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:48:01.411526 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:48:01.415492 (+ 3966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:01.417331 (+ 1839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:01.417338 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:01.417342 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:01.417523 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:48:01.418427 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:48:01.418432 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:48:01.418437 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:48:01.418878 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:48:01.419177 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:01.421658 (+ 2481us) server_negotiation.cc:299] Negotiation successful
0810 18:48:01.425074 (+ 3416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 18:48:01.439443 7636 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ae0bdcf5f5f24b18ad486c8b5031990e [term 1 FOLLOWER]: Refusing update from remote peer 0c387ebf88b84f8ea31cdab836f9f9e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:01.441946 7745 raft_consensus.cc:1261] T 00000000000000000000000000000000 P abb4ea345a894f9685874057211a0697 [term 1 FOLLOWER]: Refusing update from remote peer 0c387ebf88b84f8ea31cdab836f9f9e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:48:01.443907 7800 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0c387ebf88b84f8ea31cdab836f9f9e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae0bdcf5f5f24b18ad486c8b5031990e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 35799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:48:01.455684 7800 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0c387ebf88b84f8ea31cdab836f9f9e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abb4ea345a894f9685874057211a0697" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 44099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 18:48:01.491178 7801 sys_catalog.cc:455] T 00000000000000000000000000000000 P abb4ea345a894f9685874057211a0697 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c387ebf88b84f8ea31cdab836f9f9e1. Latest consensus state: current_term: 1 leader_uuid: "0c387ebf88b84f8ea31cdab836f9f9e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c387ebf88b84f8ea31cdab836f9f9e1" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46487 } } peers { permanent_uuid: "ae0bdcf5f5f24b18ad486c8b5031990e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 35799 } } peers { permanent_uuid: "abb4ea345a894f9685874057211a0697" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 44099 } } }
I20260810 18:48:01.491551 7801 sys_catalog.cc:458] T 00000000000000000000000000000000 P abb4ea345a894f9685874057211a0697 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:48:01.446766 7680 master_runner.cc:186] Error getting master registration for 127.4.228.254:46487: OK, instance_id { permanent_uuid: "0c387ebf88b84f8ea31cdab836f9f9e1" instance_seqno: 1786387679679741 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 18:48:01.493431 7680 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 18:48:01.499105 7793 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae0bdcf5f5f24b18ad486c8b5031990e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c387ebf88b84f8ea31cdab836f9f9e1. Latest consensus state: current_term: 1 leader_uuid: "0c387ebf88b84f8ea31cdab836f9f9e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c387ebf88b84f8ea31cdab836f9f9e1" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46487 } } peers { permanent_uuid: "ae0bdcf5f5f24b18ad486c8b5031990e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 35799 } } peers { permanent_uuid: "abb4ea345a894f9685874057211a0697" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 44099 } } }
I20260810 18:48:01.499514 7793 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae0bdcf5f5f24b18ad486c8b5031990e [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:48:01.499614 7856 mvcc.cc:205] Tried to move back new op lower bound from 7317043943134482432 to 7317043942873260032. Current Snapshot: MvccSnapshot[applied={T|T < 7317043943134482432}]
I20260810 18:48:01.512107 7863 mvcc.cc:205] Tried to move back new op lower bound from 7317043943134482432 to 7317043942873260032. Current Snapshot: MvccSnapshot[applied={T|T < 7317043943134482432}]
W20260810 18:48:01.522539 7857 catalog_manager.cc:1558] T 00000000000000000000000000000000 P abb4ea345a894f9685874057211a0697: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:48:01.522709 7857 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:48:01.526189 7793 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae0bdcf5f5f24b18ad486c8b5031990e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0c387ebf88b84f8ea31cdab836f9f9e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c387ebf88b84f8ea31cdab836f9f9e1" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46487 } } peers { permanent_uuid: "ae0bdcf5f5f24b18ad486c8b5031990e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 35799 } } peers { permanent_uuid: "abb4ea345a894f9685874057211a0697" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 44099 } } }
I20260810 18:48:01.526584 7793 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae0bdcf5f5f24b18ad486c8b5031990e [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:48:01.535048 7800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c387ebf88b84f8ea31cdab836f9f9e1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0c387ebf88b84f8ea31cdab836f9f9e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c387ebf88b84f8ea31cdab836f9f9e1" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46487 } } peers { permanent_uuid: "ae0bdcf5f5f24b18ad486c8b5031990e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 35799 } } peers { permanent_uuid: "abb4ea345a894f9685874057211a0697" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 44099 } } }
I20260810 18:48:01.536194 7800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c387ebf88b84f8ea31cdab836f9f9e1 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:01.536897 7800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c387ebf88b84f8ea31cdab836f9f9e1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0c387ebf88b84f8ea31cdab836f9f9e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c387ebf88b84f8ea31cdab836f9f9e1" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46487 } } peers { permanent_uuid: "ae0bdcf5f5f24b18ad486c8b5031990e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 35799 } } peers { permanent_uuid: "abb4ea345a894f9685874057211a0697" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 44099 } } }
I20260810 18:48:01.538306 7800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c387ebf88b84f8ea31cdab836f9f9e1 [sys.catalog]: This master's current role is: LEADER
I20260810 18:48:01.540380 7873 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 18:48:01.541970 7801 sys_catalog.cc:455] T 00000000000000000000000000000000 P abb4ea345a894f9685874057211a0697 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0c387ebf88b84f8ea31cdab836f9f9e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c387ebf88b84f8ea31cdab836f9f9e1" member_type: VOTER last_known_addr { host: "127.4.228.254" port: 46487 } } peers { permanent_uuid: "ae0bdcf5f5f24b18ad486c8b5031990e" member_type: VOTER last_known_addr { host: "127.4.228.253" port: 35799 } } peers { permanent_uuid: "abb4ea345a894f9685874057211a0697" member_type: VOTER last_known_addr { host: "127.4.228.252" port: 44099 } } }
I20260810 18:48:01.542272 7801 sys_catalog.cc:458] T 00000000000000000000000000000000 P abb4ea345a894f9685874057211a0697 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:48:01.542848 7881 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 18:48:01.545955 7839 catalog_manager.cc:1347] Generated new cluster ID: 5c62f989bcd94dffa849e56af2d48901
I20260810 18:48:01.547508 7839 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:48:01.607766 7839 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:48:01.610376 7839 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:48:01.631531 7839 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c387ebf88b84f8ea31cdab836f9f9e1: Generated new TSK 0
I20260810 18:48:01.632817 7839 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 18:48:01.769316 7676 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:01.753459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47438 (local address 127.4.228.253:35799)
0810 18:48:01.753680 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:01.753687 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:01.765876 (+ 12189us) server_negotiation.cc:407] Connection header received
0810 18:48:01.766685 (+ 809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:01.766693 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:01.766810 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:01.766900 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:01.768901 (+ 2001us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47438: BlockingRecv error: recv got EOF from 127.0.0.1:47438 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260810 18:48:01.771996 7790 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:01.759527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33682 (local address 127.4.228.254:46487)
0810 18:48:01.759749 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:01.759756 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:01.770085 (+ 10329us) server_negotiation.cc:407] Connection header received
0810 18:48:01.770408 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:01.770415 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:01.770559 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:01.770632 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:01.771690 (+ 1058us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33682: BlockingRecv error: recv got EOF from 127.0.0.1:33682 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260810 18:48:01.772195 7785 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:01.753566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60588 (local address 127.4.228.252:44099)
0810 18:48:01.753768 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:01.753775 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:01.769333 (+ 15558us) server_negotiation.cc:407] Connection header received
0810 18:48:01.770442 (+ 1109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:01.770452 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:01.770563 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:01.770641 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:01.771924 (+ 1283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60588: BlockingRecv error: recv got EOF from 127.0.0.1:60588 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260810 18:48:01.781041 7676 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:01.777251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47446 (local address 127.4.228.253:35799)
0810 18:48:01.777489 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:01.777496 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:01.778119 (+ 623us) server_negotiation.cc:407] Connection header received
0810 18:48:01.778553 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:01.778562 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:01.778722 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:01.778862 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:01.780713 (+ 1851us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47446: BlockingRecv error: recv got EOF from 127.0.0.1:47446 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
W20260810 18:48:01.792943 7676 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:01.788841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47452 (local address 127.4.228.253:35799)
0810 18:48:01.789088 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:01.789095 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:01.791103 (+ 2008us) server_negotiation.cc:407] Connection header received
0810 18:48:01.791413 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:01.791420 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:01.791540 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:01.791666 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:01.792667 (+ 1001us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47452: BlockingRecv error: recv got EOF from 127.0.0.1:47452 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
W20260810 18:48:01.804920 7676 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:01.802334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47466 (local address 127.4.228.253:35799)
0810 18:48:01.802586 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:01.802593 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:01.803173 (+ 580us) server_negotiation.cc:407] Connection header received
0810 18:48:01.803412 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:01.803420 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:01.803538 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:01.803668 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:01.804622 (+ 954us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47466: BlockingRecv error: recv got EOF from 127.0.0.1:47466 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260810 18:48:01.820178 7676 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:01.817281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47470 (local address 127.4.228.253:35799)
0810 18:48:01.817549 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:01.817557 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:01.818240 (+ 683us) server_negotiation.cc:407] Connection header received
0810 18:48:01.818496 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:01.818503 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:01.818611 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:01.818731 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:01.819906 (+ 1175us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47470: BlockingRecv error: recv got EOF from 127.0.0.1:47470 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
W20260810 18:48:01.842123 7676 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:01.838566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47476 (local address 127.4.228.253:35799)
0810 18:48:01.838816 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:01.838824 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:01.839564 (+ 740us) server_negotiation.cc:407] Connection header received
0810 18:48:01.840168 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:01.840176 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:01.840335 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:01.840475 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:01.841854 (+ 1379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47476: BlockingRecv error: recv got EOF from 127.0.0.1:47476 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
W20260810 18:48:01.866634 7676 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:01.863838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47478 (local address 127.4.228.253:35799)
0810 18:48:01.864059 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:01.864067 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:01.864769 (+ 702us) server_negotiation.cc:407] Connection header received
0810 18:48:01.865013 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:01.865021 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:01.865135 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:01.865263 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:01.866326 (+ 1063us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47478: BlockingRecv error: recv got EOF from 127.0.0.1:47478 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260810 18:48:01.907477 7676 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:01.904687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47486 (local address 127.4.228.253:35799)
0810 18:48:01.904936 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:01.904944 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:01.905568 (+ 624us) server_negotiation.cc:407] Connection header received
0810 18:48:01.905882 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:01.905889 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:01.906007 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:01.906160 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:01.907184 (+ 1024us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47486: BlockingRecv error: recv got EOF from 127.0.0.1:47486 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
W20260810 18:48:01.983191 7676 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:01.980028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47496 (local address 127.4.228.253:35799)
0810 18:48:01.980274 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:01.980281 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:01.980967 (+ 686us) server_negotiation.cc:407] Connection header received
0810 18:48:01.981436 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:01.981442 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:01.981548 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:01.981671 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:01.982927 (+ 1256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47496: BlockingRecv error: recv got EOF from 127.0.0.1:47496 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
W20260810 18:48:02.117632 7676 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:02.115690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47510 (local address 127.4.228.253:35799)
0810 18:48:02.115913 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:02.115918 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:02.116271 (+ 353us) server_negotiation.cc:407] Connection header received
0810 18:48:02.116397 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:02.116403 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:02.116506 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:02.116612 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:02.117395 (+ 783us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47510: BlockingRecv error: recv got EOF from 127.0.0.1:47510 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260810 18:48:02.381325 7676 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:48:02.378919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47518 (local address 127.4.228.253:35799)
0810 18:48:02.379250 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:02.379264 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:02.379770 (+ 506us) server_negotiation.cc:407] Connection header received
0810 18:48:02.380051 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:02.380057 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:02.380143 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:02.380274 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:48:02.381118 (+ 844us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47518: BlockingRecv error: recv got EOF from 127.0.0.1:47518 (error 108)
Metrics: {"server-negotiator.queue_time_us":126}
I20260810 18:48:02.525169 7857 catalog_manager.cc:1259] Loaded cluster ID: 5c62f989bcd94dffa849e56af2d48901
I20260810 18:48:02.525301 7857 catalog_manager.cc:1552] T 00000000000000000000000000000000 P abb4ea345a894f9685874057211a0697: loading cluster ID for follower catalog manager: success
I20260810 18:48:02.528270 7857 catalog_manager.cc:1574] T 00000000000000000000000000000000 P abb4ea345a894f9685874057211a0697: acquiring CA information for follower catalog manager: success
I20260810 18:48:02.530265 7857 catalog_manager.cc:1602] T 00000000000000000000000000000000 P abb4ea345a894f9685874057211a0697: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 18:48:02.550861 7873 catalog_manager.cc:1259] Loaded cluster ID: 5c62f989bcd94dffa849e56af2d48901
I20260810 18:48:02.551014 7873 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ae0bdcf5f5f24b18ad486c8b5031990e: loading cluster ID for follower catalog manager: success
I20260810 18:48:02.553747 7873 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ae0bdcf5f5f24b18ad486c8b5031990e: acquiring CA information for follower catalog manager: success
I20260810 18:48:02.555827 7873 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ae0bdcf5f5f24b18ad486c8b5031990e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 18:48:02.907464 7982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:02.898667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47534 (local address 127.4.228.253:35799)
0810 18:48:02.899339 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:02.899346 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:02.899773 (+ 427us) server_negotiation.cc:407] Connection header received
0810 18:48:02.899997 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:02.900002 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:02.900192 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:02.900380 (+ 188us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:02.901192 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:02.902303 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:02.904431 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:02.904956 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:02.905095 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:02.905740 (+ 645us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:02.906513 (+ 773us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:02.906903 (+ 390us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:02.906966 (+ 63us) server_negotiation.cc:299] Negotiation successful
0810 18:48:02.907061 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":328,"threads_started":1}
I20260810 18:48:02.928050 7988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:02.920626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60594 (local address 127.4.228.252:44099)
0810 18:48:02.921154 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:02.921161 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:02.921599 (+ 438us) server_negotiation.cc:407] Connection header received
0810 18:48:02.921823 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:02.921829 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:02.922039 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:02.922244 (+ 205us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:02.922697 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:02.923660 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:02.925037 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:02.925661 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:02.925798 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:02.926531 (+ 733us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:02.927299 (+ 768us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:02.927452 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:02.927619 (+ 167us) server_negotiation.cc:299] Negotiation successful
0810 18:48:02.927718 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":242,"threads_started":1}
I20260810 18:48:02.946903 7988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:02.939206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60610 (local address 127.4.228.252:44099)
0810 18:48:02.939611 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:02.939628 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:02.940644 (+ 1016us) server_negotiation.cc:407] Connection header received
0810 18:48:02.940804 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:02.940811 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:02.940925 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:02.941022 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:02.941876 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:02.942865 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:02.944788 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:02.945332 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:02.945450 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:02.946058 (+ 608us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:02.946218 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:02.946318 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:02.946525 (+ 207us) server_negotiation.cc:299] Negotiation successful
0810 18:48:02.946617 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220}
I20260810 18:48:02.947540 7982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:02.935528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47542 (local address 127.4.228.253:35799)
0810 18:48:02.935733 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:02.935742 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:02.940492 (+ 4750us) server_negotiation.cc:407] Connection header received
0810 18:48:02.940833 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:02.940843 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:02.940948 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:02.941014 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:02.941704 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:02.942764 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:02.945175 (+ 2411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:02.945844 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:02.945978 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:02.946634 (+ 656us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:02.946780 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:02.946850 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:02.947074 (+ 224us) server_negotiation.cc:299] Negotiation successful
0810 18:48:02.947159 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 18:48:02.949347 7996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:48:02.935428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33688 (local address 127.4.228.254:46487)
0810 18:48:02.941127 (+ 5699us) server_negotiation.cc:206] Beginning negotiation
0810 18:48:02.941134 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:48:02.941153 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:48:02.941273 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:48:02.941280 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:48:02.941538 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:48:02.941795 (+ 257us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:48:02.942366 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:02.943309 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:02.945433 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:48:02.945841 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:48:02.945983 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:48:02.946572 (+ 589us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:48:02.947264 (+ 692us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:48:02.947399 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 18:48:02.947575 (+ 176us) server_negotiation.cc:299] Negotiation successful
0810 18:48:02.949014 (+ 1439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5567,"thread_start_us":5463,"threads_started":1}
I20260810 18:48:02.961562 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 7456
I20260810 18:48:02.962867 7555 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:48:03.080435 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 7456
W20260810 18:48:03.116695 7714 connection.cc:535] server connection from 127.0.0.1:60544 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 18:48:03.117163 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 7571
I20260810 18:48:03.118420 7661 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:48:03.233718 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 7571
I20260810 18:48:03.269806 5011 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu pid 7680
I20260810 18:48:03.271327 7772 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 18:48:03.386767 5011 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTAuB_V/build/asan/bin/kudu with pid 7680
2026-08-10T18:48:03Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (6167 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (12727 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (415380 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST